跨国深谈

延迟 (latency)

15分钟阅读

跨国深谈← 返回←延迟 (latency)
延迟

延迟 (latency)

概念
本站收录 8 集 · 14 条金句 · 关联 10 个

集里怎么说它

① 提到它的金句

14 条

所以那个系统在几年后产生了一个芯片,其能效比当时的 CPU 和 GPU 高 30 到 80 倍,而且延迟也低得多,比如延迟降低了 20 到 30 倍。
And so that system produced a chip a couple of years later that was 30 to 80 times more energy efficient than CPUs and GPUs of the day, and also much, much lower latency, like 20 to 30x lower latency.
—— Jeff Dean · [08:12]

指向原始笔记的链接

有图谱和没有图谱相比，我们在 token、轮次和延迟上看到了巨大的改进。
With and without graph, we see massive improvement in tokens, turns, and latency.
—— Uday Kiran Medisetty · [10:08]

指向原始笔记的链接

你可能能做到质量,但成本和延迟你会吃亏,只有建立自己的研究团队、训练自己的模型才行。
You can probably get the quality, but the cost and latency you’re going to lose out on, and only by building your own research team and training your own models.
—— Cliff Obrecht · [16:39]

指向原始笔记的链接

所以一切对我来说，当你对信息和各种工作的相关信息有无限的需求时，其核心是一个围绕质量、成本和延迟的优化问题。
So everything, to me, when you have an infinite appetite for information and relevant information for all kinds of work, it’s at its core an optimization problem around quality, cost, and latency.
—— Parag · [14:22]

指向原始笔记的链接

网关的核心处，是四样东西之间的博弈：可用性、延迟、你的护栏，以及成本。
And right at the heart of the gateway is a fight between four things. It’s availability, latency, your guardrails, and costs.
—— Kanish Manuja · [01:08]

指向原始笔记的链接

我们希望随着时间的推移，一百万 token 在智能水平、成本、延迟等方面，感觉起来像五万 token。
We want a million tokens to feel like 50,000 tokens in terms of intelligence, cost, latency, et cetera, over time.
—— Alexander Whedon · [31:20]

指向原始笔记的链接

早期,我们的延迟从两秒漂移到了大约两秒半,而那半秒钟几乎让产品里的所有指标都崩了。
Early on, our latency drifted from two seconds to about two and a half seconds, and that half second tanked basically every metric in the product.
—— Paula Dozsa · [03:39]

指向原始笔记的链接

延迟不只是你最后检查的一个数字,它实际上就是产品,我们把管线的每个阶段分开来测量,因为「感觉慢」是没有用的。
Latency isn’t just a number you check at the end, it’s actually the product, and we measure every stage of the pipeline separately because it feels slow is useless.
—— Paula Dozsa · [04:54]

指向原始笔记的链接

我们观察到的一件事是，对于 Fable 级别的 LLM，它们的延迟大约每月改善 2 倍，这是非常非常快的。
So one thing that we saw is that for Fable class LLMs, their latency is improving by around 2x per month, which is very, very fast.
—— Jay · [17:35]

指向原始笔记的链接

但我认为我们将会看到延迟战争也真正升温——响应越快，你能解锁的、感觉像一个 if-else 语句的魔力就越多，越好。
But I think we’re going to start to see like latency wars really heat up as well, which is like the faster the response, the more like more magic you can unlock that feels like an if else statement, the better.
—— Clara Vo · [22:42]

指向原始笔记的链接

通过为你提供互联网上几乎任何页面的内容，我们给搜索调用增加的额外延迟为零。
We add zero extra latency to our search call by providing you the contents of pretty much any page on the internet.
—— Jakub Hojsan · [06:26]

指向原始笔记的链接

护栏的一个缺点是它总会增加延迟。如果现在我们能以非常低延迟的方式加入护栏，那会非常酷、非常强大。
One of the downsides of guardrails is that it always adds latency. If now we can add them in a really low latency way, that’s really cool and powerful.
—— 嘉宾 · [17:20]

指向原始笔记的链接

超过某个点之后，你会有点陷入收益递减：成本和延迟在增加，但输出的质量并没有显著变好。
Past a certain point, you kind of hit diminishing returns where your cost and latency are increasing, but the quality of the outputs are not significantly better.
—— Sydney · [00:57]

指向原始笔记的链接

它们实际上非常在意尾部延迟，这很有意思，因为人并不总是关心 P999 的 S3 延迟，但智能体确实在意，并且会被它卡住。
They actually care a lot about tail latencies, which is interesting, where people don’t always care about the P999 S3 latency, like agents do care and get blocked by that.
—— Matt Garman · [08:25]

指向原始笔记的链接

② 出现在这些集

8 集

③ 关联

点进去有真内容 —— 本页主要出口

智能体 · OpenAI · 微调 · 护栏 · LLM · 思维链 · 推理能力 · 评估 · Anthropic · Sarah Wang

反向链接