跨国深谈

预训练 (pre-training)

14分钟阅读

跨国深谈← 返回←预训练 (pre-training)
预训

预训练 (pre-training)

概念
本站收录 6 集 · 5 条金句 · 关联 10 个

集里怎么说它

① 提到它的金句

5 条

在这个中间过程中，既然每个人都有非常相似的预训练数据，那后训练就是如今他们做出大差异的地方。
where like post-trading, but middle course of this is more of everyone have very similar pre-training data, is that post-training is where they make a big difference nowadays.
—— Chip Huyen · [15:10]

指向原始笔记的链接

是的，我认为这是一个好问题，对开发者来说宽容的看法是，如果有一件事你真的不想搞乱，那就是预训练，因为它只是比你做的其他任何训练过程都要昂贵几个数量级。
Yeah, I think it’s a good question and the charitable take for developers is that if there’s one thing you really don’t want to mess with, it is pre-training because this is just orders of magnitude more expensive than every other training procedure that you do.
—— Adam Gleave · [74:22]

指向原始笔记的链接

并不清楚的是,用新架构和新范式预训练一个新模型的第一个周期,第一个周期就奏效是疯狂的。
It’s not clear that the first cycle of pre-training a new model with a new architecture and a new paradigm, the first cycle of that working is insane.
—— Justin Johnson · [22:32]

指向原始笔记的链接

据我们所知，我们是唯一做过稳健的数百万 token 预训练的人。
As far as we know, we are the only people to do robust multi-million token pre-training.
—— Alexander Whedon · [11:26]

指向原始笔记的链接

研究在很大程度上表明，你在后训练阶段创造价值的能力，受限于你所做过的预训练。
And so research has largely suggested that your ability to create value in the post-training phase is limited by the pre-training that you’ve done.
—— Alexander Whedon · [20:31]

指向原始笔记的链接

② 出现在这些集

6 集

③ 关联

点进去有真内容 —— 本页主要出口

后训练 · 智能体 · Anthropic · OpenAI · 合成数据 · Google · 推理 · Palantir · Transformer · Meta

反向链接