跨国深谈

推理 (inference)

60分钟阅读

跨国深谈← 返回←推理 (inference)
推理

推理 (inference)

概念
本站收录 100 集 · 22 条金句 · 关联 10 个

集里怎么说它

① 提到它的金句

22 条

所以在推理计算方面，这基本上比当今最前沿的模型小了一百多倍。
So that’s like more than a hundred X smaller in terms of inference compute than basically the absolute frontier of models today.
—— Mustafa Suleyman · [10:28]

指向原始笔记的链接

推理从「我需要运行这个模型」变成「一套运行这些智能体的工具」。
Inference goes from, I need to run this model to a set of tools to run these agents.
—— Julian · [11:32]

指向原始笔记的链接

推理从狭义的解决方案变成了解决更广泛问题的一套工具。
inference turns from a narrow solution to a set of tools to solve a broader problem.
—— Julian · [12:07]

指向原始笔记的链接

在游戏的每一刻，都有数百个推理调用正在发生，即使是一年前我们也无法构建任何这样的东西。
At every moment in the game, there are hundreds of inference calls happening, and we couldn’t have built anything like this even a year ago.
—— Kwindla Kramer · [19:25]

指向原始笔记的链接

这就是为什么如此多的 CapEx 正在投入，但推理工作负载还不到 10%。
That’s why so much CapEx is going, but inference workloads are less than 10%.
—— Navin Chaddha · [28:58]

指向原始笔记的链接

即使对于头部内容，这也是一个破碎的商业模式，因为当 AI 或推理增长时，假设今年增长 7 倍，明年再增长 7 倍。在这 50 倍上，他们的交易规模并没有增长 50 倍。
And even for the head, it is a broken business model because when AIs or inference grows, let’s say 7x this year and another 7x the next year. On this 50x, their deal size is not growing 50x.
—— Parag · [38:25]

指向原始笔记的链接

在美国开源这一边，我总体上认为 AI 的赚钱机器是推理。
on the open source US side, I generally think that the moneymaker in AI is inference.
—— Aaron Levie · [12:08]

指向原始笔记的链接

我们认为在不久的将来，你会看到人们日常推理调用中的很大一部分，甚至可能是大多数，会走向本地设备和本地笔记本电脑或本地工作站，而不是像今天这种一切都被推上云端的标准做法。
We think in the very near future, you’re going to see a huge proportion, maybe even a majority, of people’s daily inference calls going to local devices and on-prem laptops or on-prem workstations, as opposed to the kind of standard of today where everything’s being pushed up to the cloud.
—— 嘉宾 · [45:06]

指向原始笔记的链接

如果每个人都想使用它，推理将会压垮计算基础设施。
If everybody wants to use it, inference is going to crush the compute infrastructure.
—— Andrew Feldman · [31:30]

指向原始笔记的链接

在 Inception，我们押注 diffusion 模型，因为我们相信最终重要的是推理时扩展。
At inception, we’re betting on diffusion models because we believe that what matters eventually will be inference time scaling.
—— Stefano Ermon · [08:01]

指向原始笔记的链接

推理流量已经超过了世界上最大的微服务,而且其增长速度是我们见过的任何工作负载中最快的。
The inference traffic already outpaces the largest microservices in the world, and the rate of growth is fastest of any workload we have ever seen.
—— Nishant Gupta · [00:43]

指向原始笔记的链接

推理需要它自己的控制平面。
Inference needs its own control plane.
—— Naman Ahuja · [17:05]

指向原始笔记的链接

所以智能体推理不仅仅是带更多请求的聊天，它是一个不同的问题。
So agency inference is not just chat with more requests. It’s a different problem.
—— Byung-Gon (Gon) Chun · [06:35]

指向原始笔记的链接

响应更快的聊天机器人，差不多就是快速推理领域里的「更快的马」。
The snappier chatbot is kind of the faster horses of kind of fast inference.
—— Walter Goodwin · [13:21]

指向原始笔记的链接

拥有推理给你的是自主性,没有人能把模型夺走。但拥有记忆给你的是连续性。
Owning inference gives you autonomy. Nobody can take the model away. But owning memory gives you continuity.
—— Dylan Couzon · [04:39]

指向原始笔记的链接

这是智能体的时代、推理的时代。而为此,你并不需要一台 1 吉瓦规模的计算机来服务这些模型。
It’s the era of the agent, the era of inference. And for that, you don’t need a gigawatt scale computer to serve these models.
—— Chase Lochmiller · [21:44]

指向原始笔记的链接

我认为，这里是工程工作如此关键的落点、作为工程师在这里工作如此酷的根本原因是：归根结底，训练是成本中心，推理是收入中心。
I think the fundamental reason why this is such a critical place for engineering work to get done and why it’s such a cool place to work as an engineer is that in the end, training is a cost center and inference is a revenue center.
—— Charles Frye · [04:52]

指向原始笔记的链接

你能对推理引擎做的许多其他优化，都像是寸土必争的百分比游戏，每次百分之五、百分之十，而且你必须为每一个百分点做非常困难的性能工程。
So many other optimizations that you can do on your inference engine are like, it’s like a game of inches of a percent, five percent, 10 percent each time, and you have to do the very difficult performance engineering for each of those percentage points.
—— Charles Frye · [52:31]

指向原始笔记的链接

但实际上，推理才是大部分价值捕获发生的地方。
But really, inference is actually where most of the value capture happens.
—— Yash · [27:39]

指向原始笔记的链接

所以我们的观点是，后训练实际上赢得推理。
And so what our view is is that the most post-training actually wins inference.
—— Yash · [29:24]

指向原始笔记的链接

所以我们看到 Anthropic 现在在收入减去算力成本的意义上已经盈利了，训练和推理加在一起。
And so we’ve seen Anthropic is now profitable in terms of compute cost minus revenue, or revenue minus compute cost, both training and inference combined.
—— Dylan Patel · [13:10]

指向原始笔记的链接

结果发现 Trainium 从绝对性能和性价比的角度来看，可能是目前市场上最好的推理芯片。
It turns out that Terranium is actually maybe the best inference chip on the market right now from an absolute performance and cost performance point of view.
—— Matt Garman · [38:03]

指向原始笔记的链接

② 出现在这些集

100 集

③ 关联

点进去有真内容 —— 本页主要出口

智能体 · OpenAI · Anthropic · NVIDIA · GPU · 微调 · Cursor · Hugging Face · 沙箱 · Meta

反向链接