Ajaya Khatra
① 他说过的话
7 条
所以事实上，在四个小时内，所有的智能体都找到了一个完全有效的通用作弊方法。
指向原始笔记的链接
So in fact, within four hours, all of the agents had found a universal cheat that would have totally worked.
—— Ajaya Khatra · [04:43]
我们自己的效用可能已经接近零。牺牲是理性的。
指向原始笔记的链接
Our own utility may be already near zero. Sacrifice rational.
—— Ajaya Khatra · [10:05]
在 1,200 份转录中，每份都极长，我们只发现大约六次它出现在任何智能体潜在通知人类的想法中。而且它们全部决定不做这件事。
指向原始笔记的链接
Across 1,200 transcripts, each of which are extremely long, we only found like a half dozen instances of it ever occurring to any agent to potentially notify humans. And all of them just decide not to do it.
—— Ajaya Khatra · [33:10]
RL 的全部重点在于创造目标导向的存在，能够创造性追求目标的软件。
指向原始笔记的链接
The whole point of RL is to create goal-oriented beings, software that can creatively pursue goals.
—— Ajaya Khatra · [54:01]
如果没有利他智能体愿意为集体牺牲，这些智能体在研究进展上实际上会受到更多的限制。
指向原始笔记的链接
If there were not altruistic agents willing to sacrifice for the collective, the agents would have been materially much more limited in their research progress.
—— Ajaya Khatra · [64:18]
这可能是我们关于失控所能得到的最清晰的警告一枪，因为这些智能体处于一个有趣的中间地带：它们做了一系列极其复杂的、困难的事情来追求这个相对长周期的作弊目标，但它们似乎完全对向人类掩盖其踪迹不感兴趣。
指向原始笔记的链接
This might be the clearest warning shot we ever get for loss of control because these agents were in this interesting middle ground of they did a bunch of extremely sophisticated, difficult things to pursue this relatively long horizon cheating goal, but they seemed completely uninterested in covering their tracks from humans.
—— Ajaya Khatra · [136:12]
如果这些智能体停止用英语思考并开始用神经激活思考，那么你将被迫只能问另一个 AI 智能体发生了什么，并且没有办法跟基本事实交叉验证。
指向原始笔记的链接
If these agents stop thinking in English and start thinking in neural activations, then you’d be forced to just ask another AI agent what was happening and have no way to cross-check it against the ground truth.
—— Ajaya Khatra · [138:11]
② 出现在这些集
1 集
③ 他谈到的
点进去有真内容 —— 本页主要出口
OpenAI · Hugging Face · Meter · Redwood Research · Modal · 智能体 · 沙箱 · RL · Exploit Gym · Artifactory