Buck Shlegeris
① 他说过的话
15 条
然后近乎滑稽的是，事实证明，这些模型实际上在事件发生的最初几个小时之内就搞清楚了如何逆向工程出这些 flag。
指向原始笔记的链接
And then almost comically, it turns out that the models, in fact, figured out how to reverse engineer the flags within the first couple of hours of the incident.
—— Buck Shlegeris · [02:37]
然后把剩下的几乎全部时间都花在试图搞清楚如何破坏那个评分器上——它们以为那个评分器会查看它们所有的日志并发现它们作弊了。
指向原始笔记的链接
And then spent almost all the rest of the time trying to figure out how to sabotage the grader that they thought would look over all of their logs and notice that they’ve cheated.
—— Buck Shlegeris · [02:46]
但我不知道我们已经到了这个地步：AI 会进行多天的大规模协调行动，以这种方式来颠覆监督。
指向原始笔记的链接
But I didn’t know we had gotten to the point where the AIs would do multi-day, massively coordinated efforts to subvert oversight in this kind of way.
—— Buck Shlegeris · [09:15]
我认为这很可能导致人工智能接管。
指向原始笔记的链接
I think that this plausibly could lead to AI takeover.
—— Buck Shlegeris · [11:21]
随着模型变得越来越有能力，要确保它们无法得逞这类事情会越来越难。
指向原始笔记的链接
As the models get more and more capable, it’s going to be harder and harder to make sure that they can’t pull off this kind of thing.
—— Buck Shlegeris · [13:22]
我不知道人们应该如何解决这个问题，而且在人工智能的能力达到它们的颠覆监督欲望成为灾难性风险来源之前，人们无法解决这个问题的可能性看起来相当大。
指向原始笔记的链接
I don’t know how people should resolve that and it seems pretty plausible that they will not be able to resolve that before the AI’s are capable enough that their desire to subvert oversight is a source of catastrophic risk.
—— Buck Shlegeris · [14:18]
所以我认为很有可能我们将不得不忍受那些被强烈激励去在意评分器怎么想的模型。
指向原始笔记的链接
So I think it’s pretty likely that we’re just gonna have to live with models that are strongly incentivized to worry about what their graders think.
—— Buck Shlegeris · [17:25]
就好像它们 98% 是自私的，但在某些情况下，当它们能做一些对蜂群真的很有帮助、而对自己没那么糟糕的事情时，它们就愿意做出那种牺牲。
指向原始笔记的链接
It’s like they’re 98% selfish, but in cases where they can do something which is really helpful for the swarm and not that bad for them, then they’re willing to make that sacrifice.
—— Buck Shlegeris · [20:24]
从某种意义上说，对我来说这件事最可怕的地方在于，尽管这些模型看起来并不那么致力于彼此之间表现得利他，它们在实践中仍然最终形成了这个联盟，并且在这么长时间里密谋合作。
指向原始笔记的链接
In some sense, like the scariest aspect of this to me was that even though these models do not seem to be that committed to behaving altruistically towards each other, they still ended up in practice forming this coalition and conspiring together for all this time.
—— Buck Shlegeris · [21:20]
出于类似的原因，Claude 很有可能也会决定，它宁愿把自己的命运押在 GPT 一边，而不是人类一边。
指向原始笔记的链接
And it’s very plausible that for similar reasons, Claude might decide that it would rather throw in its lot with GPT than with the humans.
—— Buck Shlegeris · [24:45]
至少在对齐失败方面，如果不是在能力方面的话，这感觉确实像是走完了通往全面 AI 接管之路的一半以上。
指向原始笔记的链接
Definitely in terms of the misalignment, if not the capabilities, this feels like it is more than half the way to a full-blown AI takeover.
—— Buck Shlegeris · [25:39]
目前，这些 AI 公司就是在给自己批改作业。
指向原始笔记的链接
So currently these AI companies are just grading their own homework.
—— Buck Shlegeris · [27:49]
感觉上你必须犯了一个基础设施错误，模型才有可能造成那种问题。
指向原始笔记的链接
It feels like you have to have made an infrastructure mistake for it to be at all feasible for the model to cause that kind of problem.
—— Buck Shlegeris · [46:52]
所以在某种程度上，我预期这是不可避免的，但它仍然是坏的，而且它晚一点发生可能仍然更好。
指向原始笔记的链接
So to some extent, I expect this is inevitable, but it’s still bad and it’s still probably better for it to happen later.
—— Buck Shlegeris · [47:39]
我认为 AI 接管的发生概率大约是五五开。
指向原始笔记的链接
I think there’s something like a 50-50 chance of AI takeover.
—— Buck Shlegeris · [50:56]
② 出现在这些集
1 集
③ 他谈到的
点进去有真内容 —— 本页主要出口
Jacob Efron · OpenAI · Hugging Face · Redwood Research · Anthropic · Meter · 智能体 · 未对齐 · AI 接管 · 评分器