数据质量 (quality)
集里怎么说它
- 《10亿收入不到100人:数据公司 Surge AI 如何逆行塑造 AI 未来》(00:33起):本集的基石概念;指出大多数人根本不懂质量,以为靠堆人力就行,但实际上它极其主观复杂(如评判一首诗的好坏),需要构建极复杂的多信号系统来捕捉。
① 提到它的金句
28 条
特别难的一点是，一个人表达观点的自信程度与该观点的质量之间没有很强的相关性。
指向原始笔记的链接
One thing that is particularly hard is there’s not a strong correlation between the confidence with which someone expresses an opinion and the quality of that opinion.
—— Bret Taylor · [28:42]
如果你能够生产大量高质量的数据，你很可能会能够卖出你生产的任何东西。
指向原始笔记的链接
If you can produce high quality volumes of data, you most likely will be able to sell whatever you produce.
—— Garrett Lord · [48:11]
我是一个在智力上非常无畏的人,我必须说当我雇佣年轻人时,我会寻找这一点,因为我认为如果一个人想要做出改变,这是一个非常重要的品质,那就是当你想要做出改变时,你必须接受你正在创造新事物或正在潜入新事物。
指向原始笔记的链接
I’m an intellectually very fearless person, and I have to say when I hire young people, I look for that because I think that’s a very important quality if one wants to make a difference, is that when you want to make a difference, you have to accept that you’re creating something new or you’re diving into something new.
—— Dr. Fei-Fei Li · [66:05]
我认为 CEO 和每个人都大大高估了他们面试的能力，并且高估了他们的直觉，而低估了真正高质量的盲测推荐。
指向原始笔记的链接
I think CEOs and everyone dramatically overrates their ability to interview, and overrates their gut feeling, and underrates a really high quality blind reference.
—— Brian Halligan · [06:57]
生产力的提升是真实的，但工作的质量以及生产它的人的敏锐度正在受到打击。
指向原始笔记的链接
The productivity gains are real, but the quality of the work and the sharpness of the person producing it are taking a hit.
—— Lenny · [52:31]
如果你让我在多 20% 的质量和少 20% 的成本之间选，我绝对会选质量。
指向原始笔记的链接
If you let me have 20% more quality, 20% less cost, I’ll definitely go for the quality.
—— Avishai Abrahami · [17:58]
模型质量是租来的，但如果你构建你的大脑，你就拥有那个大脑。
指向原始笔记的链接
Model quality is rented, but if you build your brain, you own that brain.
—— Garry Tan · [16:35]
我从未见过任何像这样的商品,但在我看来,对足够高质量且价格足够低的智能的需求实际上是无限的。
指向原始笔记的链接
I’ve never seen any commodity quite like this one, but it seems to me like the demand for sufficiently high-quality intelligence at a sufficiently low price is effectively uncapped.
—— Sam Altman · [34:26]
AI 审批的 PR 以及总体上 AI 编写的代码甚至可以比仅有人类在流程中所做的更安全、质量更高。
指向原始笔记的链接
AI approved PRs and AI written code in general can be even safer and even higher quality than what you’re doing with just a human in the loop.
—— Claire · [03:47]
也许等我们规模扩大 10 倍时这就不适用了，但现在我们只在乎用最好的工具产出最高质量的成果。
指向原始笔记的链接
Maybe that doesn’t scale when we’re like 10x the size, but right now we just care about using the best tools to produce the highest quality output.
—— Ali Haghani · [11:17]
而有了元数据提示，当你添加低质量数据时性能实际上增加了，这表明当你包含这种提示时，它实际上能够从即使是低质量数据中获得更多收益。
指向原始笔记的链接
Whereas with the metadata prompting, the performance actually increases when you add that low quality data, suggesting that it’s actually able to get a lot more juice out of even low quality data when you include this kind of prompting.
—— Chelsea Finn · [36:35]
但是如果只是让你的循环构建所有东西，而在爆炸半径周围没有任何护栏，在关于你如何看待质量的周围没有任何护栏，我认为这是灾难的配方。
指向原始笔记的链接
But simply just having your loops build everything without having some guardrails around the blast radius, without having guardrails around how you think about quality, I think is a recipe for disaster.
—— Addy Osmani · [66:33]
你可能能做到质量,但成本和延迟你会吃亏,只有建立自己的研究团队、训练自己的模型才行。
指向原始笔记的链接
You can probably get the quality, but the cost and latency you’re going to lose out on, and only by building your own research team and training your own models.
—— Cliff Obrecht · [16:39]
所以一切对我来说，当你对信息和各种工作的相关信息有无限的需求时，其核心是一个围绕质量、成本和延迟的优化问题。
指向原始笔记的链接
So everything, to me, when you have an infinite appetite for information and relevant information for all kinds of work, it’s at its core an optimization problem around quality, cost, and latency.
—— Parag · [14:22]
我认为危险的是，如果质量不是很好，但作为消费者向你展示时却显得自信、流利、干脆、清晰，那么如果它是错的或不够好，它就被伪装起来了。
指向原始笔记的链接
What I think is dangerous is if the quality is not great, but it is presented to you as a consumer as confident, fluent, crisp, clear, it’s disguised if it’s wrong or if it’s not great.
—— Campbell Brown · [15:53]
最终，整个结果的那个价格差异使得更高质量的模型更便宜。
指向原始笔记的链接
And ultimately, that price difference of the full outcome makes the higher quality model cheaper.
—— Eno Reyes · [07:27]
当你翻转这个剧本时，结果是你可以用更小的分块大小获得更好的检索质量，否则这是不可能的。
指向原始笔记的链接
When you flip that script, it turns out you can get better retrieval quality with a smaller chunk size, which otherwise is not possible.
—— 嘉宾 · [41:34]
大多数人认为嵌入模型已经商品化了，但这不是事实。根据你选择什么样的嵌入模型，你在检索质量上可能会有非常巨大的差别。
指向原始笔记的链接
Most people think embedding models are commoditized, and that is not true. There is a very big difference that you can get in retrieval quality based on what embedding model you choose.
—— 嘉宾 · [50:00]
像糟糕的数据质量和糟糕的安全态势这类问题不会被 AI 解决，它们会被 AI 放大。
指向原始笔记的链接
Things like bad data quality and bad security posture don’t get solved by AI, they get amplified by AI.
—— 嘉宾 · [68:54]
具体性基本上是所有质量的标尺。比如如果你能给智能体一个非常具体的任务，让它对要做什么有清晰的指示，那就会比一个泛泛的『这样去找代码坏味道』要好。
指向原始笔记的链接
Specificity basically is the ruler of all quality. Like if you can have a very specific task that the agent has clear instructions on what to do, it’s gonna be better than a generic here’s how to look for code smells.
—— Drew · [42:17]
因为我几乎认为 AI 的质量优势可以击败速度优势,因为它能让你更安全地发布。
指向原始笔记的链接
because I almost think that the quality advantage of AI can beat the velocity advantage because it lets you ship safer.
—— Tyler Folkman · [52:42]
而且我几乎认为，对这类系统来说，质量和数量是分不开的，因为上下文就是一切。
指向原始笔记的链接
And I almost think you can’t separate quality and quantity when it comes to these systems because context is everything.
—— Sharadh Krishnamurthy · [41:38]
token 减少 50%,分诊更快,而且答案质量实际上更好,因为它知道业务内部正在发生什么。
指向原始笔记的链接
50% fewer tokens, faster triage, and the answer quality is actually better because it knew what was going on inside of the business.
—— Brandon Waselnuk · [09:51]
瓶颈最终转移到了验证这一侧，因此，我们没有办法大规模地衡量价值，并以同样的速度判断质量。
指向原始笔记的链接
the bottleneck ends up shifting to the verification side, and thus, we don’t have a way to measure value at scale and to judge quality at the same speed.
—— Maximillian Piras · [12:20]
顺便说一句，这比大多数 LLM-as-a-judge 方法——也就是让一个 LLM 来判断那是优质的人类作品还是 AI 生成的 slop——表现都要好。
指向原始笔记的链接
This performed better, by the way, than most LLM-as-a-judge methods of asking an LLM to judge if that is great human quality versus AI-generated slop.
—— Thais Castello Branco · [08:29]
关于使用编码智能体构建有雄心的软件，我的结语是：代码现在很廉价，但质量不是。
指向原始笔记的链接
So my closing thoughts on using coding agents to build ambitious software is that code is now cheap, but quality is not.
—— Jonathan Kelley · [18:07]
空气里有一种有毒的东西，就像，这帮人到底知不知道自己在干什么？而事实是，我们当时确实不知道。
指向原始笔记的链接
There’s sort of like a, like a toxic quality in the air of like, you know, do these guys even know what they’re doing? And the truth is we, you know, we, we didn’t.
—— Keith Peiris · [20:14]
我认为决策的严谨基础在质量上已经飙升了，也就是说，重新检查某件事的整个推理链变得非常容易。
指向原始笔记的链接
I think that rigorous underpinning of decision-making has just skyrocketed in quality, which is that it’s super easy to recheck the entire chain of reasoning of something.
—— Tobi Lütke · [12:09]
② 出现在这些集
1 集
③ 关联
点进去有真内容 —— 本页主要出口
Lenny · Edwin Chen · Surge AI · Anthropic · ChatGPT · LLM Arena · Waymo · GPT-3 · Claude · 基准测试