跨国深谈

护栏 (guardrails)

Properties1
aliasesguardrails, safeguards

54分钟阅读

跨国深谈← 返回←护栏 (guardrails)
护栏

护栏 (guardrails)

概念 · 又名 guardrails / safeguards
本站收录 90 集 · 20 条金句 · 关联 10 个

集里怎么说它

① 提到它的金句

20 条

实际上我认为人们会在国防行业感到惊讶，我们的行业周围可能比大多数商业行业有更多的护栏和负责任的 AI 努力。在美国这里有联邦法规禁止某些事情，而在商业领域人们为了安全问题可能只是激增并去做。
I actually think people would be surprised in the defense industry that there is probably more guardrails and responsible AI efforts around our industry than most commercial industries. There are federal regulations here in The US that prohibit certain things that in the commercial space people might just surge and go do in terms of safety issues.
—— Chris Benson · [22:38]

指向原始笔记的链接

这个问题在于前沿模型在自动化红队测试方面极其糟糕,因为它们内置了大量的保障措施。
the issue with this is that frontier models are extremely bad at automated red teaming because they have a lot of safeguards built into them.
—— Zico Kolter · [09:59]

指向原始笔记的链接

而现在我们实际上发现，我们设下的所有那些护栏，正在拖累这些聪明得多的 AI。
Now what we’re finding actually is that all those guardrails we put in place are holding these much more intelligent AIs back.
—— Jon Noronha · [15:46]

指向原始笔记的链接

你应该描述任务、描述护栏、描述退出标准,然后就让模型自己开干,过一会儿再回来。
You want to describe the task, you want to describe the guardrails, you want to describe the exit criteria, and then just go with the model cook and come back in a little bit.
—— Boris Cherny · [15:19]

指向原始笔记的链接

归根结底，所有的闭源专有模型 API，它们的护栏有点武断，但也很难强制执行。
In the end, it’s about all the closed proprietary model APIs, their guardrails are a little bit arbitrary, but also very difficult to enforce.
—— Simon Mo · [32:24]

指向原始笔记的链接

我经常在前沿方面遇到模型，它们不让我做我想做的事情，因为我很多时候试图做新奇的事情，我到处都撞到护栏。
I run into models all the time on the frontier side that won’t let me do things I want to do because I’m trying to do novel things a lot of the time and I hit guardrails all over the place.
—— Chris Benson · [48:22]

指向原始笔记的链接

是的，很多人都在谈论围绕这些东西的护栏，但我认为从基础设施角度，成本方面也需要护栏，因为你不会想要不小心启动了什么东西，结果突然收到一笔巨额账单，只因为某个智能体做了一个不一定正确的决定。
Yeah, I mean, a lot of people talk about guardrails around these things, but I think from infrastructure, guardrails around the cost as well, because you don’t want to accidentally spin up something that suddenly you have a huge bill for because some agent made a decision that was not necessarily the right decision.
—— Sean Falconer · [22:56]

指向原始笔记的链接

我认为人们已经了解到，无论 AI 实验室在这些东西周围设置了什么护栏，你都不能依赖模型来阻止自己或理解上下文。
And I think what people have learned is that regardless of guardrails that the AI labs are putting around these things, you can’t rely on models to stop themselves or to understand context.
—— Nick Warner · [07:16]

指向原始笔记的链接

但是如果只是让你的循环构建所有东西，而在爆炸半径周围没有任何护栏，在关于你如何看待质量的周围没有任何护栏，我认为这是灾难的配方。
But simply just having your loops build everything without having some guardrails around the blast radius, without having guardrails around how you think about quality, I think is a recipe for disaster.
—— Addy Osmani · [66:33]

指向原始笔记的链接

如果你自动启动这台机器，其影响将远超今天，因为这会变成在没有治理机制的情况下维持自动演进的技能——而治理机制才是真正为你的组织设置护栏的东西。
If you automatically start this machine, the impacts will be way more than it is today because what I shared earlier is gonna be just maintaining auto-evolving skills without that governance in place that actually put the guardrails for your organization.
—— Imad Touil · [19:48]

指向原始笔记的链接

网关的核心处，是四样东西之间的博弈：可用性、延迟、你的护栏，以及成本。
And right at the heart of the gateway is a fight between four things. It’s availability, latency, your guardrails, and costs.
—— Kanish Manuja · [01:08]

指向原始笔记的链接

你可以把这些护栏直接内置到钱包里，而你作为人类不必逐笔批准每一笔交易，因为那样根本无法扩展
You sort of build these guardrails into the wallet, and you don’t have to, as a human, approve every single transaction, because that would just not scale
—— Harshal Bhangale · [09:32]

指向原始笔记的链接

智能体现在可以持有自己的钱，自主地花这些钱，但要在你设定的护栏之内。
The agents can now hold their money, spend that money autonomously, but within the guardrails that you set.
—— Harshal Bhangale · [19:38]

指向原始笔记的链接

你给一个 LLM 一个目标，它会在护栏之内竭尽所能去解决那个目标。
You give an LLM a goal, it will do everything it can within guardrails to solve that goal.
—— 嘉宾 · [19:26]

指向原始笔记的链接

但是这里的教训，这是元层面的教训，护栏是不够的。
But the learning, here’s the meta learning, guardrails aren’t enough.
—— 嘉宾 · [29:30]

指向原始笔记的链接

所以这个循环基本上应该像一个棘轮：尝试一个改动，给它打分，只有当记分卡改善且护栏完好时才保留它。
So the loop should basically be like a ratchet. Try a change, score it, keep it only if the scorecard improves and guardrails whole.
—— Erina Karati · [18:07]

指向原始笔记的链接

我觉得很讽刺的是,你不能鼓励一个组织毫无护栏地 all-in——他们就是这么做的——然后,另一方面,又对组织里大量的垃圾内容有意见。你必须选定你的车道。
I find it ironic that you can’t encourage an organization to go all-in, which is what they did, with no guardrails around it, and then, on the other hand, have a problem with the massive amounts of slop in your organization. You have to pick your lane.
—— Faisal Masud · [05:45]

指向原始笔记的链接

我记得那大概是一个 30000 行的 ClaudeMD，里面全是各种护栏，以及关于他应该如何与客户打交道的建议。
I think it’s like a 30,000 line ClaudeMD of all these guardrails and advice on how he should be operating with customers.
—— Matt Lawler · [11:37]

指向原始笔记的链接

护栏的一个缺点是它总会增加延迟。如果现在我们能以非常低延迟的方式加入护栏，那会非常酷、非常强大。
One of the downsides of guardrails is that it always adds latency. If now we can add them in a really low latency way, that’s really cool and powerful.
—— 嘉宾 · [17:20]

指向原始笔记的链接

这就是为什么围绕 LLM 的那些护栏真的不是长期来看正确的解决方案。
This is why all the guardrails around LLMs are really not long-term going to be the right solution.
—— Sam Newman · [96:53]

指向原始笔记的链接

② 出现在这些集

90 集

③ 关联

点进去有真内容 —— 本页主要出口

智能体 · OpenAI · Anthropic · 沙箱 · Codex · MCP · ChatGPT · harness · 推理 · Hugging Face

反向链接