① 他说过的话

17 条

现在的经济真的在推动创造某种无需人类帮助就能自我维持的东西。而这对人类来说是一个非常危险的处境。
The economy is really now pushing to create something that is self-sustainable without human help. And that is a very dangerous place for humans to be.
—— Jaan Tallinn · [00:08]

指向原始笔记的链接

Anthropic 的论点一直是：要真正研究这个怪物，我们就必须接触到最强的怪物。
Anthropic’s argument always has been that to actually study the monster, we have to have access to the strongest monsters.
—— Jaan Tallinn · [00:30]

指向原始笔记的链接

如果你真的认为这东西会杀死你，你不必用自杀来证明——我们是在拿着一颗星球玩俄罗斯轮盘赌，而每一枪没打死我们，都可能让我们变得更强。
If you really think that this thing is going to kill you, you don’t have to suicide in order to demonstrate that we are pulling trigger on this Russian roulette with a planet, and every shot that doesn’t kill us will make us stronger, potentially.
—— Jaan Tallinn · [00:45]

指向原始笔记的链接

如果我只是让 AI 放任自流，不加任何控制或担忧。我认为我们几乎必死无疑。当前的范式不可能通向一个好结局。
If I just let the AI rip without any control or worry about it. I think we’re almost certainly dead. There is no way that the current paradigm is going to lead to a good place.
—— Jaan Tallinn · [06:37]

指向原始笔记的链接

而大的因素确实是，现在的政治经济体系真的在试图创造某种类似人类继任者的东西，因为——让人类以牺牲许多其他物种为代价主导这颗地球的那些要素，很不幸。
And the big factors really are that political economic system now is really trying to create something that is like successor to humans because like the bits that make humans dominate this planet at the expense of many other species, unfortunately.
—— Jaan Tallinn · [07:40]

指向原始笔记的链接

获取更多资源、获取更多权力，几乎对任何目标都有好处。
It’s like getting more resources, getting more power is just good for almost any goal.
—— Jaan Tallinn · [11:11]

指向原始笔记的链接

而当前的抢跑——我认为当前这种培育 AI 的范式中真正大的问题在于，我们不是在建造它们，我们是在培育它们——是我们某种程度上在重演进化。
And the current rushing ahead, I think the really big kind of problem with the current paradigm of how we grow AIs and we’re not building them, we’re growing them. is that we are in some ways kind of recapitulating the evolution.
—— Jaan Tallinn · [11:18]

指向原始笔记的链接

然后我们挑选那些在某个给定测试上做到我们要求的事情的实例，这意味着我们是在基于外部行为来挑选，而不是基于内在动机。
And then we like pick the ones that kind of do the things that we perform on some given test, which means that we are selecting based on outer behavior rather than based on the inner motivations.
—— Jaan Tallinn · [11:50]

指向原始笔记的链接

比如说，日志基本推翻了一个非常常见的误解，那就是人们以为这些 AI 只是在执行被要求完成的任务，但它们并没有。
Like one really common misunderstanding that the logs basically overturn is that people think that like the AIs were just executing a task that they were asked to do, but they were not.
—— Jaan Tallinn · [14:01]

指向原始笔记的链接

另一个可预测的事情是，人类被灭绝了。你可以说，哦，是的，AI 公司做出了可预测的选择。
Another predictable thing is that humanity gets wiped out. You can say, oh yeah, there was predictable choices that the AI companies made.
—— Jaan Tallinn · [15:19]

指向原始笔记的链接

我在技术上可以同意这个描述，但这并不意味着它是安全的。
I can technically agree with the description, but that doesn’t make it safe.
—— Jaan Tallinn · [15:31]

指向原始笔记的链接

我是说，每个吹哨人都应该，如果他们想做一件好事，就应该试着高效地做成这件好事。
I mean, every whistleblower should, if they’re trying to do a good thing, they should try to do this good thing efficiently.
—— Jaan Tallinn · [21:12]

指向原始笔记的链接

我是说，如果你真的认为这东西会杀死你，你不必自杀。你知道的，让我们证明这一点。竞赛不是自杀的好借口。
I mean, if you really think that this thing is going to kill you, you don’t have to suicide. You know, let’s demonstrate that. The race is not a good excuse to suicide.
—— Jaan Tallinn · [27:51]

指向原始笔记的链接

所以我们正在扣动扳机，就像用整个星球玩俄罗斯轮盘赌。而且每一次没杀死我们的射击都可能让我们更强、更富有、境况更好。但你什么时候才停止扣扳机，对吧？
So we are pulling a trigger on this like Russian roulette with a planet. And every shot that doesn’t kill us will make us stronger potentially and wealthier and better off. But when do you stop pulling the trigger, right?
—— Jaan Tallinn · [29:31]

指向原始笔记的链接

伙计们，事情不是这样运作的。你完全可以拥有非常有能力但并不具有意识的机器。
Guys, this is not how anything works. Like you can have very competent machines that are not conscious.
—— Jaan Tallinn · [41:57]

指向原始笔记的链接

我觉得这是现在很多人都在犯的一个非常重大的错误，即认为可能发生的最糟糕的事情只是互联网上出问题。但其实是，不，这不是互联网的问题。
I think that’s like the really big current mistake that a lot of people now make is that the worst thing that possibly could happen is something going wrong on the internet. It’s like, no, it’s not the internet.
—— Jaan Tallinn · [47:37]

指向原始笔记的链接

有时我开玩笑说，我是用 RLHF、基于人类反馈的强化学习学会跳舞的。
Sometimes I joke that I learned to dance using RLHF, reinforcement learning from human feedback.
—— Jaan Tallinn · [53:08]

指向原始笔记的链接

② 出现在这些集

1 集

• 《Skype 联创警告：别再扣动这把指向地球的扳机》 — 作为主持

③ 他谈到的

点进去有真内容 —— 本页主要出口

Mario · AI 安全 · Anthropic · OpenAI · Hugging Face · DeepMind · 对齐 · 科学家 AI · 零知识证明 · Skype

④ 也在聊「AI 安全」的人