跨国深谈

思维链 (chain of thought)

18分钟阅读

跨国深谈← 返回←思维链 (chain of thought)
思维

思维链 (chain of thought)

概念
本站收录 12 集 · 5 条金句 · 关联 10 个

集里怎么说它

① 提到它的金句

5 条

因为我们发现的是，虽然我们通常可以越狱模型，很难让它在思维链中推理时，对自己即将做的坏事闭嘴。
Because what we found is that even though we can usually jailbreak the model, it’s really hard to get it to shut up about the evil thing that it’s about to do when it’s reasoning in the chain of thought.
—— Adam Gleave · [33:19]

指向原始笔记的链接

但每次你基于对思维链的观察进行干预,你都在隐式地施加一点压力,让模型之后隐藏它的思维链。
But every time you intervene based on your observations of the chain of thought, you are implicitly applying a tiny bit of pressure for the model to then hide its chain of thought.
—— Noam Brown · [70:17]

指向原始笔记的链接

我认为安全机制是在争取时间,思维链监控之类的东西也是在争取时间,它们能告诉我们是否走在正确的道路上。但归根结底,我们确实需要解决对齐问题。
I think that the safety mechanisms bias time, and things like chain of thought monitoring bias time, and they can tell us if we’re on the right path. But at the end of the day, we really do need to solve the alignment problem.
—— Noam Brown · [74:01]

指向原始笔记的链接

已经有研究表明，如果你对思维链做大量监控，它们就开始在思维链里撒谎。
It’s been shown that if you do a lot of monitoring on a chain of thought, they start lying in their chain of thoughts.
—— Amjad Masad · [35:57]

指向原始笔记的链接

这场对话，以及我最近其他几场关于 AI 监控与控制未来的对话，得出的底线是：思维链正变得越来越不可靠。
The bottom line from this conversation and others I’ve had recently on the future of AI monitoring and control is that chain of thought is increasingly unreliable.
—— 嘉宾 · [03:03]

指向原始笔记的链接

② 出现在这些集

12 集

③ 关联

点进去有真内容 —— 本页主要出口

智能体 · OpenAI · Anthropic · Hugging Face · 沙箱 · 后训练 · RL · 微调 · 推理 · 护栏

反向链接