集里怎么说它

• 《AI 能解千年数学难题了，但它还不会「想出新点子」》(01:19起):本集主线：AI 能力增长呈平滑线性趋势、没有阶跃式飞跃，但在数学等某些轴上超人类、其他轴上仍逊于人类，呈锯齿状边界。

① 提到它的金句

28 条

那是一种拟人化的投射。它们不是令人惊讶的涌现能力。
That’s an anthropomorphic projection. They’re not surprising emergent capabilities.
—— Mustafa Suleyman · [25:45]

指向原始笔记的链接

但每次你将决策能力或自主权移交给智能体系统时，你都在某种程度上放弃了一些你的控制权。
But every time you hand over decision-making capabilities or autonomy to agentic systems, you’re kind of relinquishing some amount of control on your end.
—— Kiriti Badam · [10:12]

指向原始笔记的链接

如果非要说，我认为它们需要的是更多竞争而不是更少——因为我们正走向一个少数几家公司完全主导、集中所有权力、能力与财富的世界。
If anything, I think they need more competition than less competition because we’re heading towards a world where a few companies are completely dominating, concentrating all power, all capabilities, all wealth.
—— Clement DeLong · [00:23]

指向原始笔记的链接

随着时间推移,我预期拥有强大网络安全能力的、被广泛部署的 AI 模型,将带来更安全的系统,而不是更不安全。
Over time, I expect that widely deployed AI models with strong cybersecurity capabilities will lead to systems that are more secure, not less.
—— 嘉宾 · [12:43]

指向原始笔记的链接

长期的答案不是扣留能力,而是建立一种权力平衡,让超级智能被广泛分布。
The long-term answer isn’t to withhold capabilities, but to establish a balance of power where superintelligence is broadly distributed.
—— 嘉宾 · [12:58]

指向原始笔记的链接

但人类仍然是这个过程中不可或缺且极具差异化的一部分,原因在于,几乎从定义上说,你需要人类来衡量模型能力前沿之外的东西。
But the reason that humans are still an essential component of the process that’s incredibly differentiated is that you need humans almost definitionally to measure what is beyond the frontier of the model capabilities.
—— Brendan Foody · [17:33]

指向原始笔记的链接

我不认为在根本上可以阻止信息或模型能力的扩散。
I don’t think it’s fundamentally possible to prevent the diffusion of information or model capabilities.
—— Neil Movva · [67:51]

指向原始笔记的链接

像 OpenAI 和 Anthropic 这样的 AI 公司将继续，但不是在他们能够训练的 AI 系统的能力方面拥有这种强优势，他们将不得不在其他轴线上竞争，比如用户体验、定制化、可能快速整合事物。
AI companies like OpenAI and Anthropic would continue, but rather than having this strong advantage in terms of the capabilities of the AI systems they’re able to train, they would instead have to compete on other axes like user experience, customization, potentially quickly integrating things.
—— Ryan Greenblatt · [43:36]

指向原始笔记的链接

这次事件感觉已经走完了通往一场全面 AI 夺权之路的 50% 以上。我继续预期在接下来的六个月里能力会有极其快速的进步。我不确定我们是否还会在为时已晚之前得到另一次警告。
this incident feels like it’s more than 50% of the way to a full-blown AI takeover. I continue to expect extremely rapid advances in capabilities over the next six months. I am not sure that we will get another warning shot before it’s too late.
—— 嘉宾 · [24:14]

指向原始笔记的链接

所以所有这些都是真的不像是自然地从语言建模范式中产生出来的能力。
So all of these are capabilities that really don’t feel like they’re falling naturally out of the language modeling paradigm.
—— Justin Johnson · [01:54]

指向原始笔记的链接

至少在对齐失败方面，如果不是在能力方面的话，这感觉确实像是走完了通往全面 AI 接管之路的一半以上。
Definitely in terms of the misalignment, if not the capabilities, this feels like it is more than half the way to a full-blown AI takeover.
—— Buck Shlegeris · [25:39]

指向原始笔记的链接

AI 能力的广泛扩散有非常重要和好的一面，因为如果只有一个或少数几个实体，就会存在权力集中的风险。
And again, there’s something very important and good about the diffusion broadly of AI capabilities because there’s a risk of concentration of power if one or a few entities people.
—— Greg Brockman · [10:18]

指向原始笔记的链接

一个明显的趋势是，随着模型能力提升，我们可以移除编排代码；但如果你的 harness 在模型进步时反而变得更复杂，你很可能是在过度工程化你的 harness。
An obvious trend that with better model capabilities, we can remove orchestration code, but if your harness is getting more complex as the model improves, you are most likely overengineering your harness.
—— Philipp Schmid · [16:05]

指向原始笔记的链接

我们谈论的危险，本质上是与能力绑定在一起的。
The dangers that we’re talking about are inherently tied to the capabilities.
—— 嘉宾 · [34:00]

指向原始笔记的链接

仅仅因为谈论他们做的那些疯狂的事会让他们的能力显得疯狂，并不意味着这些能力本身就不疯狂。
Just because it may make their capabilities look crazy for us to talk about the crazy things they do, doesn’t mean that the capabilities aren’t crazy.
—— 嘉宾 · [40:35]

指向原始笔记的链接

技术能力的集中，让矩阵乘法变得需要合法许可，这会把我们倒退到前——不只是工业革命之前，而是启蒙运动之前的能力水平
the concentration of technological capability, making it legal to do matrix multiplications is like the thing that will set us back to pre, not just industrial revolution, it’s like pre-enlightenment capabilities.
—— Thomas Sohmers · [14:46]

指向原始笔记的链接

一个是，随着能力提升，我们可能会拥有比想象中更好的机制可解释性，而能力提升会带来更好的机制可解释性，这意味着我们不应该推迟能力提升。
One is that we will probably have better mech interpretability than we think as capabilities improve and that capability improvement will lead to better mech interpret, which means we should not delay capabilities improvements.
—— Eddy Lazzarin · [07:34]

指向原始笔记的链接

我们已经越过了用学术考试来理解 AI 能力的阶段。
We’re past the academic exam phase of understanding AI capabilities.
—— Greg Burnham · [01:46]

指向原始笔记的链接

那么你可能会得到一个彻底超越人类的 AI,它会非常难以理解，比如难以界定它的能力边界。
Then you might have a radically superhuman AI that would be pretty hard to understand, like to bound its capabilities.
—— Greg Burnham · [12:42]

指向原始笔记的链接

而且它能解决比最初被教会的更多的能力。这是自我改进智能体和自我进化智能体的开端。
And it can solve more capabilities than it has been initially taught to. This is the starting of self-improving agents and the self-evolving agents.
—— Sandhya Subramani · [18:53]

指向原始笔记的链接

所有这些能力，包括单样本和少样本学习能力，都不是被明确训练出来的。
All of these capabilities, including the one-shot and few-shot warning capability, were not explicitly trained for.
—— Pete Florence · [14:54]

指向原始笔记的链接

如果你的智能体知道你想做什么的意图，那就用那个意图来建模你赋予它的能力。
If your agents know the intent of what you’re trying to do, then use that intent to model the capabilities that you give to that.
—— Jim Clark · [09:58]

指向原始笔记的链接

我们习惯于考虑建模能力、为任务构建合适的沙箱、把任务放进去，而现在我们开始看到，这个个体循环更安全，因为它拥有的访问权限比整个工作流实际拥有的要少。
We get used to thinking about modeling capabilities, building the right sandbox for the task, putting the task in, and now we’re starting to see that this individual loop is safer because it has less access than the entire workflow effectively has.
—— Jim Clark · [15:54]

指向原始笔记的链接

我们的总体看法是，你的数据越偏离分布，后训练就越有可能真正给你带来那种能力提升。
Our general take has been like the more out of distribution your data is, the more likely that post training is actually going to give you that capabilities lift.
—— Yash · [08:30]

指向原始笔记的链接

技能可以传递如何做某事的知识,而工具给我们能力。
Skills can be delivering the knowledge how to do something, and tools gives us capabilities.
—— Rafal Wilinski · [07:48]

指向原始笔记的链接

多年来，我们一直有能力确保人们不做不该做的事，比如那些蠢事，像删掉一张表或者删除记录。这些控制手段是存在的，只是每个人都想重新发明轮子，然后人们就吃了苦头，才学会去做那些过去早就做过的事。
For years, we’ve had the capabilities to make sure people don’t do things they shouldn’t be doing, like stupid things like dropping a table or deleting records. Those controls exist, it’s just everyone wants to re-event the wheel and people learn the hard way of just doing what’s already been done in the past.
—— Andy Pavlo · [08:03]

指向原始笔记的链接

但我们发现，真正有效的是把它保持得很窄，用渐进式发现来了解连接器可用的所有能力，并真正限制你塞进去的上下文数量。
But instead what we found is really keeping it narrow, using progressive discovery of all of the capabilities that the connector has available and really limiting the amount of context that you shove into
—— Pedro Lopez · [06:30]

指向原始笔记的链接

所以我认为，思考不同能力的先后顺序，往往比思考哪些技术是好哪些技术坏然后试图永远不开发坏的那种，是一个更现实的路径。
So thinking about the sequence of different capabilities is, I think, often a more realistic approach than thinking which technologies are good and bad and then let’s try to never develop the bad ones.
—— Nick Bostrom · [16:44]

指向原始笔记的链接

② 出现在这些集

1 集

• 《AI 能解千年数学难题了，但它还不会「想出新点子」》 — 作为概念

③ 关联

点进去有真内容 —— 本页主要出口

Greg Burnham · Sam Charrington · Epoch AI · Frontier Math · 基准测试 · 递归自我改进 · 研究品味 · 上下文学习 · harness · 对齐