集里怎么说它

• 《把 100 万 token 用出 5 万的体验：SubQuadratic 的稀疏注意力》(00:00起):本集的出发点:注意力让每个 token 与其他所有 token 互相关照,1000 个 token 有 100 万个两两关系,成本随上下文长度平方增长——上下文翻倍,算力大约贵四倍。
• 《内存带宽才是 AI 芯片的下一个战场：Fractile 创始人 Walter 的押注》(13:54起):本集说开源中国模型前沿的注意力机制每隔几周就在变；现有快速推理芯片容量太低跑不了长上下文注意力，得切回 GPU。

① 提到它的金句

15 条

现在，我主要受限于注意力。与 tokens 或计算不同，我不能简单地增加更多的注意力。
Now, I’m primarily constrained by attention. And unlike tokens or compute, I can’t simply add more of it.
—— 嘉宾 · [21:19]

指向原始笔记的链接

如果 Transformer 是有利可图的,如果你可以花费更多的努力和资源来扩大 Transformer 以在下一个季度获胜,那么很难把大量的注意力和精力投入到可能在一两年内更好或重新定义该领域的事情上。
If Transformer is profitable and if you can spend more efforts and more resources scaling Transformer to win in the next quarter, it’s very hard to put at least a lot of attention and a lot of energy to work on something that will maybe better or maybe will redefine the field in a year or two.
—— Jerry Tworek · [13:28]

指向原始笔记的链接

就像在人类历史的绝大部分时间里，如果你是一个聪明的 20 岁年轻人，有了一个让自己的社会变得更好的想法，你把它讲给有钱人听，反应会是：你应该去关心收成。
Like for the vast majority of human history, if you were a smart 20 year old that like had an idea to like make your society better and you raise this to the people with capital, the reaction was like, you should pay attention to the harvest.
—— Max Hodak · [46:47]

指向原始笔记的链接

因为，你知道，初创公司不直接瞄准现有企业，而现有企业只是不注意。
And because, and the, and because, you know, the startups don’t aim straight at the incumbents, and the incumbents just don’t pay attention.
—— Steven Sinofsky · [49:54]

指向原始笔记的链接

如果你是一个美国人,你希望我们在汽车生意上打败中国人,你们都该密切关注——但不一定是 Tesla。
If you’re an American and you want us to beat the Chinese in the car business, you’re all going to want to pay attention, not necessarily to Tesla.
—— Jim Farley · [12:11]

指向原始笔记的链接

现在，错的人在驾驶座上。我们称他们为「AI 兴奋者」。AI 兴奋者是公司里获得所有关注的人。
Right now, the wrong people are in the driver’s seat. We call them the AI excited. The AI excited are the ones who are getting all the attention in the company.
—— Mike Lewis · [30:28]

指向原始笔记的链接

他们的动力是改善自己的利润底线，变现你的注意力，拿走你的数据，锁死它，再卖回给你。
They are incentivized to improve their bottom line, to monetize your attention, to take your data, lock it in, and sell it back to you.
—— Benjamin Guo · [04:02]

指向原始笔记的链接

有一场争夺注意力的战争。如果你造了个没人用的东西，那它就跟没被造出来一样。
There’s a war for attention. If you build something that nobody uses, it might as well have not been built.
—— Jordan Tigani · [01:00]

指向原始笔记的链接

你必须对未来有一个猜测。然后你必须有毅力和力量，在猜测错了时去注意并纠正。
You’ve got to have a guess. And then you have to have the fortitude and the strength to pay attention when your guess is wrong and correct.
—— Andrew Anagnost · [14:22]

指向原始笔记的链接

我认为那时你应该识别出那个巨大的瓶颈是什么,然后把整个公司、你全部的注意力几乎极端地、以数量级级别的专注,全部放在这一个问题上。
And I think you should at that point identify what that giant bottleneck is and then focus the whole company in your all attention almost to an extreme, like orders of magnitude of attention and focus should just go to this one issue.
—— Ali Ghodsi · [09:53]

指向原始笔记的链接

gated DeltaNet 及其衍生版本，通过这种机制，你在注意力部分花费的总时间可以减少大约 75%
gated DeltaNet and its derivative versions where you can have like a 75% decrease in the total time you’re spending on the attention portion with this mechanism.
—— Thomas Sohmers · [57:38]

指向原始笔记的链接

现在，人类才是瓶颈，因为他们注意力有限，因为有这么多并行的工作在进行。
Now, humans are the bottleneck with their limited attention because so much parallel work is going on.
—— Dario Hamidi · [34:51]

指向原始笔记的链接

我当时过度优化的是今天什么火、什么叙事是对的、我们怎么去劫持注意力，诸如此类，真的。一切的一切，就是不去做一个好产品。
I was so much optimizing for what’s hype today, what’s the right narrative, how we can hijack the attention, all these things, really. Everything instead of building a good product.
—— Alex Mashrabov · [11:54]

指向原始笔记的链接

获得关注非常容易，保持尊重却很难。
It’s very easy to get attention. It’s very hard to keep respect.
—— Andrew Huberman · [20:15]

指向原始笔记的链接

如果你无法承受这种复杂性，因为你需要他要么是恶棍要么是英雄，那你根本没有在认真看。
And if you can’t take on that complexity because you need him to be either villain or hero, you’re just not paying attention.
—— 嘉宾 · [101:40]

指向原始笔记的链接

② 出现在这些集

2 集

• 《把 100 万 token 用出 5 万的体验：SubQuadratic 的稀疏注意力》 — 作为概念
• 《内存带宽才是 AI 芯片的下一个战场：Fractile 创始人 Walter 的押注》 — 作为概念

③ 关联

点进去有真内容 —— 本页主要出口

Alexander Whedon · Walter Goodwin · SubQuadratic · Fractile · 稀疏注意力 · NVIDIA · 上下文工程 · Broadcom · 智能体 · TSMC