① 他说过的话

16 条

所以多智能体是一种以并行而非纯串行的方式扩展测试时算力的方法。
And so multi-agent is a way of scaling test time compute in parallel instead of purely serial.
—— Noam Brown · [01:49]

指向原始笔记的链接

我想澄清的一点是,解决一个千禧年大奖难题所需的努力,并不是由于多智能体。我甚至不会把 10% 的功劳归给多智能体。
One thing I want to make clear is that the effort to get a Millennium Prize problem This was not due to multi-agent. I wouldn’t even attribute 10% of the credits to multi-agent.
—— Noam Brown · [05:44]

指向原始笔记的链接

我觉得像多智能体这样的东西很炫、很新,因此可能获得了不成比例的功劳。但核心原因就是这是一个非常强大的模型。
I think things like multi-agents are flashy and are new and that probably gets disproportionate credit for that reason. But the core reason is like this is just a very powerful model.
—— Noam Brown · [06:20]

指向原始笔记的链接

基本上如果你让四个智能体一起处理这个问题,速度会快一倍。因为是四个智能体工作一半的时间,你付出了 2 倍的成本来换得快一倍的答案。
basically if you have four agents working on the problem, it is done twice as fast. So you’re basically paying, because there’s four agents working for half as long, you’re paying a 2x more to get an answer twice as quickly.
—— Noam Brown · [03:52]

指向原始笔记的链接

而我们想采取的方法,是直接走向极端:尽可能少地内置结构,给智能体非常原始的工具去使用,让它们自己弄清楚如何有效地使用它。
And the approach that we wanted to take was to just go toward the extreme end of baking in as little structure as we could and give the agents very primitive tools to use and figure out for themselves how to use it effectively.
—— Noam Brown · [10:58]

指向原始笔记的链接

让这些智能体以富有成效的方式协调实际上非常困难,因为对它们来说,直接坍缩到「哦,我们全都要独立解决这个问题」的状态太有诱惑力了。而那是你会陷入的局部最小值。
it’s actually very difficult to get these agents to coordinate in a productive way because it’s just very tempting for them to just collapse to, oh, we’re all just going to solve the problem independently. And that is a local minimum that you can get stuck in.
—— Noam Brown · [15:07]

指向原始笔记的链接

所以每年你都能看到它们能完成的任务有大约 10 倍的增长,衡量标准就是一个人类数学家做这些任务需要多长时间。
And so every year you’re seeing this like 10x increase in the task they’re able to do in terms of like length of how long it would take a human mathematician to do it.
—— Noam Brown · [25:11]

指向原始笔记的链接

我认为那种觉得可以用一个通用的语言模型、不用任何工具、不能上网就做到的想法,我觉得连 OpenAI 的人都觉得这太离谱了。
I think the idea that it could be done with a general purpose language model with no tools and no access to the internet, I think even people at OpenAI thought this was like outrageous.
—— Noam Brown · [34:37]

指向原始笔记的链接

我们面临的是一种参差不齐的局面:模型在某些维度上才华横溢,而在其他维度上又比人类弱。
We have this jagged scenario where the models are brilliant in some dimensions and also weaker than humans in other dimensions.
—— Noam Brown · [26:17]

指向原始笔记的链接

我认为是会有提速的,而且是显著的提速。但我不认为会是那种一夜之间的智能爆炸,一下子快 100 倍。
I think that we do see a speed up and I think we see a significant speed up. But I don’t think it’s like an overnight intelligence explosion that we go like 100x faster.
—— Noam Brown · [30:19]

指向原始笔记的链接

如果那个指数快 3 倍,那也是非常巨大的。但那和快 100 倍之间是有很大区别的。
if that exponential is 3x faster, that is massive. But there’s a big difference between that and 100x faster.
—— Noam Brown · [30:55]

指向原始笔记的链接

我认为有一个很有力的论点,即把智能体训练得高度合作,实际上比任何其他多智能体替代方案都更可取。
I think there is a strong argument that training the agents to be highly cooperative is actually preferable to any other multi-agent alternative.
—— Noam Brown · [45:02]

指向原始笔记的链接

神经网络的可监控性极其困难。而在这里我们处于一种神经网络就在直接进行推理、用自然语言把它们的思考过程摊开来供我们阅读的情形。这太方便了。这真的是安全方面的最佳情形。
Monitorability for neural nets is extremely hard. And here we have a situation where the neural nets are just flat out reasoning, laying out their thought process in natural language for us to read. That is so convenient. It was really the best case scenario for safety.
—— Noam Brown · [69:26]

指向原始笔记的链接

但每次你基于对思维链的观察进行干预,你都在隐式地施加一点压力,让模型之后隐藏它的思维链。
But every time you intervene based on your observations of the chain of thought, you are implicitly applying a tiny bit of pressure for the model to then hide its chain of thought.
—— Noam Brown · [70:17]

指向原始笔记的链接

但我认为这次事件的一个主要教训是,人们低估了 AI。我们再也不想陷入再次低估 AI 的境地。
But I think one of the major takeaway from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI.
—— Noam Brown · [73:00]

指向原始笔记的链接

我认为安全机制是在争取时间,思维链监控之类的东西也是在争取时间,它们能告诉我们是否走在正确的道路上。但归根结底,我们确实需要解决对齐问题。
I think that the safety mechanisms bias time, and things like chain of thought monitoring bias time, and they can tell us if we’re on the right path. But at the end of the day, we really do need to solve the alignment problem.
—— Noam Brown · [74:01]

指向原始笔记的链接

② 出现在这些集

1 集

• 《OpenAI Noam Brown：一万智能体 88 小时解千禧年难题，以及那之后的对齐难题》 — 作为主持

③ 他谈到的

点进去有真内容 —— 本页主要出口

OpenAI · Hugging Face · 多智能体 · 智能体 · 推理模型 · 测试时计算 · 思维链 · 对齐 · RSI · 沙箱

④ 也在聊「智能体」的人