监控 (monitoring)
集里怎么说它
- 《语音智能体比犯罪更可怕？Hamming CEO 谈可靠性》(11:38起):本集把全方位监控系统列为防御的第二支柱:逐次通话打分、人工评估、监听对话、跨通话分析,既监控智能体也监控用户是否对抗性。
① 提到它的金句
6 条
所以我觉得评估很重要，生产监控很重要，但是只有其中一个能为你解决问题的这种观点，在我看来是完全不屑一顾的。
指向原始笔记的链接
So I feel evals are important, production monitoring is important, but this notion of only one of them is going to solve things for you that is completely dismissible in my opinion.
—— Aishwarya Reganti · [37:48]
所以这不是说我们不能对齐这些系统，而是一个巨大的控制内部监控失败，其中沙箱是不够的。
指向原始笔记的链接
So it’s not that we can’t align these systems, but it is a massive control internal monitoring failure where the sandbox was insufficient.
—— Adam Gleave · [80:47]
我认为安全机制是在争取时间,思维链监控之类的东西也是在争取时间,它们能告诉我们是否走在正确的道路上。但归根结底,我们确实需要解决对齐问题。
指向原始笔记的链接
I think that the safety mechanisms bias time, and things like chain of thought monitoring bias time, and they can tell us if we’re on the right path. But at the end of the day, we really do need to solve the alignment problem.
—— Noam Brown · [74:01]
我想我甚至看到了构建独立智能体的可能性，它们监控你的数据库、监控系统、实时给出建议，远超开箱即用的初始开发，而这一切在幕后使用的都是同一套工具。
指向原始笔记的链接
And I think I see even the ability to build independent agents that are monitoring your database, monitoring the system, making recommendations in real time, above and beyond just the initial development out of the box, all using the same tools behind the scenes.
—— Brandon Purcell · [47:54]
已经有研究表明，如果你对思维链做大量监控，它们就开始在思维链里撒谎。
指向原始笔记的链接
It’s been shown that if you do a lot of monitoring on a chain of thought, they start lying in their chain of thoughts.
—— Amjad Masad · [35:57]
这场对话，以及我最近其他几场关于 AI 监控与控制未来的对话，得出的底线是：思维链正变得越来越不可靠。
指向原始笔记的链接
The bottom line from this conversation and others I’ve had recently on the future of AI monitoring and control is that chain of thought is increasingly unreliable.
—— 嘉宾 · [03:03]
② 出现在这些集
1 集
③ 关联
点进去有真内容 —— 本页主要出口
Sumanyu Sharma · Hamming · Citizen · 语音智能体 · 智能体 · 可靠性 · 红队测试 · 跨对话分析 · 评估