安全 (safety)
集里怎么说它
- 《OpenAI产品负责人：不想让人类成为AI的「事后想起」》(02:12起):本集说安全是产品的根本要求：Astra 6.1 因安全评测轻微回退被扣住不发，部署给十亿用户前标准近乎苛刻，安全要始终跑在能力前面一步。
① 提到它的金句
22 条
高性能机器没有心理安全感。它们是为了赢。
指向原始笔记的链接
High performance machines don’t have psychological safety. They’re about winning.
—— Keith Rabois · [74:41]
实际上我认为人们会在国防行业感到惊讶，我们的行业周围可能比大多数商业行业有更多的护栏和负责任的 AI 努力。在美国这里有联邦法规禁止某些事情，而在商业领域人们为了安全问题可能只是激增并去做。
指向原始笔记的链接
I actually think people would be surprised in the defense industry that there is probably more guardrails and responsible AI efforts around our industry than most commercial industries. There are federal regulations here in The US that prohibit certain things that in the commercial space people might just surge and go do in terms of safety issues.
—— Chris Benson · [22:38]
所以在对齐和安全中最重要的指标之一是你对你正在被测试有多了解。因为你非常了解的那一刻，我们应该停止。
指向原始笔记的链接
So so the one of the most important metrics in alignment and safety is how aware are you that you’re being tested. Because the moment that you’re very aware, we should stop.
—— Swyx · [38:45]
当我们与现场技术员合作时，他们告诉我们，如果我必须脱下安全手套来在一个小玩意上打字来使用你的小玩意，我就不会去做。
指向原始笔记的链接
When we work with field technicians, they tell us, if I have to take my safety glove off to type into a thingy to use your thingy, I’m just not going to do it.
—— Kriti Sharma · [05:16]
我对 10 年后的一个反乌托邦感到特别紧张的是我们对 AI 安全过度反应。
指向原始笔记的链接
One dystopia that I’m particularly nervous about 10 years from now is we overreact to AI safety.
—— Sam Altman · [36:57]
更难构建的能力是将那项前沿研究投入生产，并在安全关键的环境中部署它而不出现回归。
指向原始笔记的链接
The much harder muscle to build is to carry that bleeding edge research into production and deploy it in a safety-critical environment without regressions.
—— Dmitri Dolgov · [22:56]
公司如何做采购、会计、招聘、绩效评估、预算、安全和质量，就是这家公司的操作系统。
指向原始笔记的链接
How the company does purchasing and accounting and recruiting and performance reviews and budgeting and safety and quality is the operating system of the company.
—— Max Hodak · [24:00]
因为归根结底，共享出行就是价格、可靠性和安全性。就是这样。10 年前就是这样。我认为 10 年后即使有了自动驾驶汽车，也会是这样。
指向原始笔记的链接
Because ride sharing at the end of the day is price, reliability, and safety. That’s all it is. That’s what it was 10 years ago. I think that’s what it’s going to be 10 years from now, even when it’s autonomous vehicles.
—— Andrew MacDonald · [08:07]
所以我认为特别是在 Whatnot 上的信任与安全方面，占据了 40% 的员工。
指向原始笔记的链接
So I think trust and safety in particular on Whatnot is about 40% of employees.
—— Grant LaFontaine · [33:29]
交付给患者,会把你通常发布时依赖的那些常规安全网全部拿走。
指向原始笔记的链接
So shipping to patients takes away the normal safety nets you would normally ship with.
—— Jared Joselowitz · [00:58]
Docker 在过去十年解决的是可移植性——怎么把软件从你的笔记本搬到云端。我们正在把所有这些经验拿来构建一个运行时，并把它演进到如今去解决安全性问题。
指向原始笔记的链接
And what Docker solved the last decade is portability. How do you get software from your laptop to the cloud? We’re taking all of that experience and building a runtime and evolving that to now solve for safety.
—— Tushar Jain · [10:35]
是的，我们已经证明，在将安全性提高 40% 的同时，成本最多降低了 83%。
指向原始笔记的链接
Yeah, we have demonstrated as much as 83% reduction in costs as we improved safety by 40%.
—— Manoj Saxena · [32:08]
我们认为未来的国家会是那些在国防、公共安全、医疗健康等方面最早采用 AI 的国家，而且这会渗透到整个政府，然后也从底层由他们的公民使用它而兴起。
指向原始笔记的链接
We think countries of the future are gonna be the ones that adopt AI first across fence, across public safety, across healthcare, and that infiltrates all throughout the government and then also comes bottom up from their citizens utilizing it as well.
—— Jen Kha · [11:58]
神经网络的可监控性极其困难。而在这里我们处于一种神经网络就在直接进行推理、用自然语言把它们的思考过程摊开来供我们阅读的情形。这太方便了。这真的是安全方面的最佳情形。
指向原始笔记的链接
Monitorability for neural nets is extremely hard. And here we have a situation where the neural nets are just flat out reasoning, laying out their thought process in natural language for us to read. That is so convenient. It was really the best case scenario for safety.
—— Noam Brown · [69:26]
我认为安全机制是在争取时间,思维链监控之类的东西也是在争取时间,它们能告诉我们是否走在正确的道路上。但归根结底,我们确实需要解决对齐问题。
指向原始笔记的链接
I think that the safety mechanisms bias time, and things like chain of thought monitoring bias time, and they can tell us if we’re on the right path. But at the end of the day, we really do need to solve the alignment problem.
—— Noam Brown · [74:01]
我觉得当我们先担心安全，再担心我们搁置了什么、延迟的代价是什么时，我们有点本末倒置了。
指向原始笔记的链接
I think we’re putting the cart before the horse a little bit when we worry about safety before we worry about what we’re leaving on the table and what the costs of delay are.
—— Eddy Lazzarin · [02:13]
人们很容易利用我们正在讨论的这种安全诉求，钻进安全这个兔子洞，最终在安全的幌子下，碰巧设计出某种完美地微妙控制一个整个至关重要的行业的东西，而且表面上看是去中心化的，但实际上不是。
指向原始笔记的链接
It’s very easy to use the calls for safety like we’re discussing and to go down this rabbit hole of safety and end up in a place where you just by coincidence have designed something that is perfect to subtly control an entire critically important industry under the pretense of safety and with something that superficially looks decentralized but is not.
—— Eddy Lazzarin · [11:38]
今天,每个人都需要成为一名专职飞手,我经历过认证流程,还要考虑安全标准。但你可以想象,几年之后,安全将由自主系统来决定。
指向原始笔记的链接
Today, everyone needs to be a dedicated pilot. I went through a certification exercise. I need to think about the safety standards here, but you can imagine in a few years’ time, the safety is gonna be determined by the autonomous system.
—— Suchet Bargoti · [04:32]
创业梦想本身就是安全网。
指向原始笔记的链接
The entrepreneurial dream is the safety net.
—— Tomer London · [52:30]
所以一个可合并性的基准，如果我们能做出来的话，不仅仅是一种度量，它会立刻成为前沿模型的训练信号——也就是可维护性、范围纪律、回归安全性这些。
指向原始笔记的链接
So a benchmark of mergeability, if we could make one, wouldn’t only be a measurement, it would immediately become a training signal for the frontier models, which means maintainability, scope, discipline, regression safety.
—— Laurie Voss · [09:59]
我们在 API 栈上的大部分投入,实际上是投入到安全栈上的。
指向原始笔记的链接
The majority of our investment on the API stack is actually going into the safety stack.
—— Tibo Sottiaux · [34:58]
而且我认为这是 Hugging Face 事件的教训之一，即安全不只是当你即将部署它、检查它是否可以安全部署的时候才开始，而是要贯穿整个训练过程本身，那可能存在不对齐的风险。
指向原始笔记的链接
And like, I think this is one of the lessons from the Hugging Face is that safety needs to start, not just when you’re about to deploy it and in like checking that it’s safe to deploy, but throughout the training process itself, that could be a risk of misalignment.
—— Nick Bostrom · [09:14]
② 出现在这些集
1 集
③ 关联
点进去有真内容 —— 本页主要出口
Rana El-Khalyubi · Tibo Sottiaux · OpenAI · Codex · ChatGPT · Dots · Astra · 智能体 · 沙箱 · 对齐