'How marketplaces win: Liquidity, growth levers, quality, more | Benjamin Lauzier (Lyft, Thumbtack)'

Lenny Rachitsky: You wrote somewhere that creating powerful AI might be the last invention humanity ever needs to make.

Lenny Rachitsky：你在某处写道，创造强大的人工智能可能是人类需要做出的最后一项发明。

How much time do we have, Ben?

本，我们还有多少时间？

Benjamin Mann: I think 50th percentile chance of hitting some kind of superintelligence is now like 2028.

本杰明·曼：我认为 2028 年达到某种超级智能的概率是 50%。

Lenny Rachitsky: What is it that you saw at OpenAI?

Lenny Rachitsky：您在 OpenAI 看到了什么？

What'd you experience there that made you feel like, okay, we got to go do our own thing?

你在那里经历了什么让你觉得，好吧，我们必须去做我们自己的事情？

Benjamin Mann: We felt like safety wasn't the top priority there.

本杰明·曼：我们觉得安全并不是那里的首要任务。

The case for safety has gotten a lot more concrete, so superintelligence is a lot about how do we keep God in a box and not let the God out?

安全问题已经变得更加具体，因此超级智能主要是关于我们如何将上帝关在盒子里而不让上帝出去？

Lenny Rachitsky: What are the odds that we align AI correctly?

Lenny Rachitsky：我们正确调整人工智能的可能性有多大？

Benjamin Mann: Once we get to superintelligence, it will be too late to align the models.

本杰明·曼：一旦我们达到超级智能，调整模型就为时已晚。

My best granularity forecast for could we have an X-risk or extremely bad outcome is somewhere between 0 and 10%.

对于我们是否会出现 X 风险或极其糟糕的结果，我的最佳粒度预测是在 0 到 10% 之间。

Lenny Rachitsky: Something that's in the news right now is this whole Zuck coming after all the top AI researchers,

Lenny Rachitsky：现在的新闻是扎克伯格正在追杀所有顶尖的人工智能研究人员，

Benjamin Mann: We've been much less affected because people here, they get these offers and then they say, well, of course I'm not going to leave because my best case scenario at Meta is that we make money and my best case scenario at Anthropic is we affect the future of humanity.

本杰明·曼：我们受到的影响要小得多，因为这里的人们，他们收到了这些报价，然后他们说，好吧，我当然不会离开，因为我在 Meta 最好的情况是我们赚钱，而我在 Anthropic 最好的情况是我们影响人类的未来。

Lenny Rachitsky: Dario, your CEO recently talked about how unemployment might go up to something like 20%.

Lenny Rachitsky：Dario，你们的首席执行官最近谈到失业率可能会上升到 20% 左右。

Benjamin Mann: If you just think about 20 years in the future where we're way past the singularity, it's hard for me to imagine that even capitalism will look at all like it looks today.

本杰明·曼：如果你想想未来 20 年，我们已经超越了奇点，我很难想象资本主义也会像今天一样。

Lenny Rachitsky: Do you have any advice for folks that want to try to get ahead of this?

Lenny Rachitsky：对于那些想要抢占先机的人们，您有什么建议吗？

Benjamin Mann: I'm not immune to job replacement either.

本杰明·曼：我也不能幸免于工作替代。

At some point it's coming for all of us.

在某个时刻，它会降临到我们所有人身上。

Lenny Rachitsky: Today, my guest is Benjamin Mann.

莱尼·拉奇茨基：今天，我的客人是本杰明·曼。

Holy moly.

圣钼。

What a conversation.

多么好的对话啊。

Ben is the co-founder of Anthropic.

Ben 是 Anthropic 的联合创始人。

He serves as tech lead for product engineering.

他担任产品工程的技术主管。

He focuses most of his time and energy on aligning AI to be helpful, harmless, and honest.

他将大部分时间和精力集中在使人工智能变得有益、无害和诚实上。

Prior to Anthropic, he was one of the architects of GPT-3 at OpenAI.

在加入 Anthropic 之前，他是 OpenAI 的 GPT-3 的架构师之一。

In our conversation, we cover a lot of ground, including his thoughts on the recruiting battle for top AI researchers, why he left OpenAI to start Anthropic, how soon he expects we'll see AGI.

在我们的谈话中，我们讨论了很多话题，包括他对顶级人工智能研究人员招募战的想法、他为什么离开 OpenAI 创办 Anthropic、他预计我们多久才能看到 AGI。

Also, his economic touring test for knowing when we've hit AGI, why scaling laws have not slowed down and are in fact accelerating and what the current biggest bottlenecks are.

此外，他的经济巡回测试旨在了解我们何时达到通用人工智能，为什么规模法则没有放缓，实际上正在加速，以及当前最大的瓶颈是什么。

Why he's so deeply concerned with AI safety and how he and Anthropic operationalize safety and alignment into the models that they build and into their ways of working.

为什么他如此深切地关注人工智能安全，以及他和 Anthropic 如何将安全性和一致性运用到他们构建的模型和工作方式中。

Also, how the existential risk from AI has impacted his own perspectives on the world and his own life and what he's encouraging his kids to learn to succeed in an AI future.

此外，人工智能带来的生存风险如何影响了他对世界和生活的看法，以及他鼓励孩子们学习什么，以便在人工智能的未来取得成功。

Lenny Rachitsky: A huge thank you to Steve Mnich, Danielle Ghiglieri, Raph Lee, and my newsletter community for suggesting topics for this conversation.

Lenny Rachitsky：非常感谢 Steve Mnich、Danielle Ghiglieri、Raph Lee 和我的时事通讯社区为本次对话提出的主题。

If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube.

如果您喜欢这个播客，请不要忘记订阅并在您最喜欢的播客应用程序或 YouTube 中关注它。

Also, if you become an annual subscriber of my newsletter, you get a year free of a bunch of amazing products including Bolt, Linear, Superhuman, Notion, Granola, and more.

此外，如果您成为我的时事通讯的年度订阅者，您将获得一年免费的一系列令人惊叹的产品，包括 Bolt、Linear、Super human、Notion、Granola 等。

Check it out at Lennysnewsletter.com and click bundle with that I bring you Benjamin Mann.

请访问 Lennysnewsletter.com，然后单击“我为您带来本杰明·曼 (Benjamin Mann)”的捆绑包。

Lenny Rachitsky: This episode is brought to you by Sauce.

Lenny Rachitsky：这一集是由 Sauce 为您带来的。

The way teams turn feedback into product impact is stuck in the past.

团队将反馈转化为产品影响的方式仍然停留在过去。

Vague reports, static taxonomies, unactionable insights that don't move business metrics.

模糊的报告、静态的分类法、无法采取行动的见解都不会改变业务指标。

The results churn, lost deals, misgrowth.

结果是流失、交易失败、增长不良。

Sauce is the AI product co-pilot that helps CPOs and product teams uncover business impact and act faster.

Sauce 是 AI 产品副驾驶，可帮助 CPO 和产品团队发现业务影响并更快采取行动。

It listens to your sales calls, support tickets, turn reasons, and lost deals, surfacing the biggest product issues and opportunities in real time.

它会听取您的销售电话、支持请求、转机原因和失败的交易，实时显示最大的产品问题和机会。

Lenny Rachitsky: It then routes them to the right teams to turn signals into PRDs, prototypes, and even code that drives revenue retention and adoption.

Lenny Rachitsky：然后将他们路由到合适的团队，将信号转化为 PRD、原型，甚至是推动收入保留和采用的代码。

That's why Whatnot, Linktree, Incident.io, and Zip use Sauce.

这就是 Whatnot、Linktree、Incident.io 和 Zip 使用 Sauce 的原因。

One enterprise uncovered a product gap that unlocked $16 million ARR, another caught a spiking issue and prevented millions in churn.

一家企业发现了产品差距，释放了 1600 万美元的 ARR，另一家企业发现了一个峰值问题并避免了数百万人的流失。

You can too at sauce.app/lenny.

您也可以访问酱汁.app/lenny。

Sauce built for AI product teams.

为人工智能产品团队打造的酱汁。

Don't get left behind.

不要落后。

Lenny Rachitsky: This episode is brought to you by LucidLink, the storage collaboration platform.

Lenny Rachitsky：本集由存储协作平台 LucidLink 为您带来。

You've built a great product, but how you show it through video, design, and storytelling is what brings it to life.

您已经打造了一款出色的产品，但如何通过视频、设计和讲故事来展示它才是让它变得栩栩如生的关键。

If your team works with large media files, videos, design assets, layer project files, you know how painful it can be to stay organized across locations, files live in different places.

如果您的团队处理大型媒体文件、视频、设计资源、图层项目文件，您就会知道跨位置保持组织有序、文件位于不同位置是多么痛苦。

You're constantly asking, is this the latest version?

您不断地问，这是最新版本吗？

Creative work slows down while people wait for files to transfer.

当人们等待文件传输时，创造性工作就会减慢。

LucidLink fixes this.

LucidLink 修复了这个问题。

It gives your team a shared space in the cloud that works like a local drive.

它为您的团队提供了一个类似于本地驱动器的云共享空间。

Files are instantly accessible for anywhere, no downloading, no syncing, and always up to date.

文件可以在任何地方立即访问，无需下载，无需同步，并且始终是最新的。

That means producers, editors, designers, and marketers can open massive files in their native apps, work directly from the cloud, and stay aligned wherever they are.

这意味着制作人、编辑、设计师和营销人员可以在其本机应用程序中打开大量文件，直接从云端工作，并且无论身在何处都可以保持一致。

Teams at Adobe, Shopify, and top creative agencies use LucidLink to keep their content engine running fast and smooth.

Adobe、Shopify 和顶级创意机构的团队使用 LucidLink 来保持其内容引擎快速流畅地运行。

Try it for free at lucidlink.com/lenny.

请访问 lucidlink.com/lenny 免费试用。

That's L-U-C-I-D-L-I-N-K dot com slash Lenny.

这是 L-U-C-I-D-L-I-N-K . com 斜线 Lenny。

Lenny Rachitsky: Ben, thank you so much for being here.

Lenny Rachitsky：Ben，非常感谢你来到这里。

Welcome to the podcast.

欢迎来到播客。

Benjamin Mann: Thanks for having me.

本杰明·曼：谢谢你邀请我。

Great to be here, Lenny.

很高兴来到这里，莱尼。

Lenny Rachitsky: I have a billion and one questions for you.

Lenny Rachitsky：我有十亿零一个问题要问你。

I'm really excited to be chatting.

我真的很高兴能聊天。

I want to start with something that's very timely, something that's happening this week.

我想从本周发生的一件非常及时的事情开始。

Something that's in the news right now is this whole Zuck coming after all the top AI researchers offering them $100 million signing bonuses, $100 million comp.

现在的新闻是，扎克伯格向所有顶级人工智能研究人员提供了 1 亿美元的签约奖金和 1 亿美元的补偿金。

He's poaching from all the top AI labs.

他从所有顶级人工智能实验室挖人。

I imagine this something you're dealing with.

我想这是你正在处理的事情。

I'm just curious, what are you seeing inside Anthropic and just what's your take on the strategy?

我只是很好奇，您在 Anthropic 内部看到了什么以及您对该战略有何看法？

Where do you think things go from here?

您认为事情接下来会走向何方？

Benjamin Mann: Yeah, I mean I think this is a sign of the times.

本杰明·曼：是的，我的意思是我认为这是时代的标志。

The technology that we're developing is extremely valuable.

我们正在开发的技术非常有价值。

Our company is growing super, super fast.

我们公司的发展速度非常非常快。

Many of the other companies in the space are growing really fast.

该领域的许多其他公司都在快速增长。

And at Anthropic, I think we've been maybe much less affected than many of the other companies in the space because people here are so mission oriented and they stay because...

在 Anthropic，我认为我们受到的影响可能比该领域的许多其他公司要小得多，因为这里的人是如此以使命为导向，他们留下来是因为……

They get these offers and then they say, "Well, of course I'm not going to leave because my best case scenario at Meta is that we make money and my best case at Anthropic is we affect the future of humanity and try to make AI flourish and human flourishing go well." To me, it's not a hard choice.

他们收到这些邀请后会说：“好吧，我当然不会离开，因为我在 Meta 最好的情况是我们赚钱，而我在 Anthropic 最好的情况是我们影响人类的未来，并努力让人工智能蓬勃发展，让人类繁荣发展。”对我来说，这并不是一个艰难的选择。

Other people have different life circumstances and it makes it a much harder decision for them.

其他人有不同的生活环境，这使得他们的决定变得更加困难。

For anybody who does get those mega offers and accepts them, I can't say I hold it against them when they accept it, but it's definitely not something that I would want to take myself if it came to me.

对于那些确实得到了这些巨额报价并接受它们的人来说，我不能说当他们接受它时我会反对他们，但如果它降临到我身上，我绝对不想接受这样的事情。

Lenny Rachitsky: Yeah.

莱尼·拉奇茨基：是的。

We're going to talk about a lot of this stuff that you've mentioned.

我们将讨论您提到的很多此类内容。

In terms of the offers do you think, is this a real number that you're seeing this $100 million signing bonus, is that a real thing?

就报价而言，您认为这是一个真实的数字吗？您看到的 1 亿美元签约奖金是真的吗？

I don't know if you've actually seen that.

我不知道你是否真的见过。

Benjamin Mann: If you just think about the amount of impact that individuals can have on a company's trajectory, in our case, we are selling hotcakes and if we get a 1 or 10 or 5% efficiency bonus on our inference stack, that is worth an incredible amount of money.

本杰明·曼（Benjamin Mann）：如果你只考虑个人对公司发展轨迹的影响程度，在我们的例子中，我们卖的是烤饼，如果我们的推理堆栈获得 1%、10% 或 5% 的效率加成，那就值一笔令人难以置信的钱。

And so to pay individuals like $100 million over four year package, that's actually pretty cheap compared to the value created for the business.

因此，在四年内向个人支付 1 亿美元的费用，与为企业创造的价值相比，这实际上相当便宜。

I think we're just in an unprecedented era of scale and it's only going to get crazier actually.

我认为我们正处于一个前所未有的规模时代，而且实际上只会变得更加疯狂。

If you extrapolate the exponential on how much companies are spending, it's like 2X a year roughly in terms of CapEx, and today we're maybe in the globally $300 billion range, the entire industry spending on this, and so numbers like 100 million are a drop in the bucket.

如果你用指数来推断公司的支出金额，就资本支出而言，每年大约是 2 倍，而今天我们可能在全球 3000 亿美元的范围内，整个行业在这方面的支出，所以像 1 亿这样的数字只是九牛一毛。

But if you go a few years out, a couple more doublings, we're talking about trillions of dollars and at that point it's just really hard to think about these numbers.

但如果再过几年，再翻一番，我们谈论的是数万亿美元，到那时，真的很难考虑这些数字。

Lenny Rachitsky: Along these lines, something that a lot of people feel with AI progress is that we're hitting plateaus in many ways that it feels like newer models are just not as smart as previous leaps.

Lenny Rachitsky：沿着这些思路，很多人对人工智能进步的感受是，我们在很多方面都处于停滞状态，感觉新模型并不像以前的飞跃那么聪明。

But I know you don't believe this.

但我知道你不相信这一点。

I know you don't believe that we've hit plateaus on scaling loss.

我知道您不相信我们的规模损失已达到稳定水平。

Talk about just what you're seeing there and what you think people are missing.

谈谈你在那里看到了什么以及你认为人们错过了什么。

Benjamin Mann: It's kind of funny because this narrative comes out every six months or so and it's never been true, and so I kind of wish people would have a little bit of a bullshit detector in their heads when they see this.

本杰明·曼：这有点有趣，因为这个故事大约每六个月就会出现一次，但它从来都不是真的，所以我希望人们在看到这个时，脑子里能有一点废话探测器。

I think progress has actually been accelerating where if you look at the cadence of model releases, it used to be once a year and now with the improvements in our post-training techniques, we're seeing releases every month or three months, and so I would say progress is actually accelerating in many ways, but there's this weird time compression effect.

我认为进展实际上一直在加速，如果你看看模型发布的节奏，过去是每年一次，现在随着我们训练后技术的改进，我们每个月或三个月都会看到一次发布，所以我想说，进展实际上在很多方面都在加速，但有一种奇怪的时间压缩效应。

Dario compared it to being in a near light speed journey where a day that passes for you is like five days back on earth and we're accelerating.

达里奥将其比作一次近光速旅行，对你来说，一天的时间就像回到地球上的五天一样，而且我们正在加速。

The time dilation is increasing.

时间膨胀正在增加。

Benjamin Mann: And I think that's part of what's causing people to say that progress is slowing down, but if you look at the scaling laws, they're continuing to hold true.

本杰明·曼：我认为这就是人们说进步正在放缓的部分原因，但如果你看看缩放定律，它们仍然有效。

We did kind of need this transition from normal pre-training to reinforcement learning scaling up to continue the scaling laws, but I think it's kind of like for semiconductors where it's less about the density of transistors that you can fit on a chip and more about how many flops can you fit in a data center or something.

我们确实需要从正常的预训练到强化学习的转变，以继续扩展法则，但我认为这有点像半导体，它不太关心芯片上可以安装的晶体管的密度，而是更多地关心数据中心或其他东西可以安装多少次触发器。

You have to change the definition around a little bit to keep your eye on the prize.

你必须稍微改变一下定义，才能把注意力集中在奖品上。

But yeah, this is one of the few phenomena in the world that has held across so many orders of magnitude.

但是，是的，这是世界上为数不多的跨越如此多个数量级的现象之一。

It's actually pretty surprising that it is continuing to hold.

事实上，它能继续保持下去确实令人惊讶。

To me, if you look at fundamental laws of physics, many of them don't hold across 15 orders of magnitude, so it's pretty surprising.

对我来说，如果你观察一下基本物理定律，就会发现其中许多定律并不适用于 15 个数量级，所以这是相当令人惊讶的。

Lenny Rachitsky: It boggles the the mind.

Lenny Rachitsky：这令人难以置信。

What you're saying essentially is we're seeing newer models being released more often, and so we're comparing it to the last version and we're just not seeing as much advance.

您所说的本质上是我们看到更新的型号发布得更频繁，因此我们将其与上一个版本进行比较，但我们只是没有看到那么多的进步。

But if you go back and it was like a model released once a year, it was a huge leap, and so people are missing that.

但如果你回顾过去，它就像每年发布一次的模型一样，这是一个巨大的飞跃，所以人们会错过这一点。

We're just seeing many more iterations.

我们只是看到更多的迭代。

Benjamin Mann: I guess, to be a little bit more generous to the people saying things are slowing down.

本杰明·曼：我想，对那些说事情正在放缓的人要宽容一点。

I think that for some tasks we are saturating the amount of intelligence needed for that task, maybe to extract information from a simple document that already has form fields on it or something like it's just so easy that okay, yeah, we're already at 100% and there's this great chart on Our World in Data that shows that when you release a new benchmark within six to 12 months, it immediately gets saturated.

我认为，对于某些任务，我们正在饱和该任务所需的智能量，也许是从已经有表单字段的简单文档中提取信息，或者类似的东西，这非常简单，好吧，是的，我们已经达到 100%，我们的数据世界上有一张很棒的图表，显示当您在 6 到 12 个月内发布新基准时，它会立即饱和。

And so maybe the real constraint is how can we come up with better benchmarks and better ambition of using the tools that then reveals the bumps in intelligence that we're seeing now.

因此，也许真正的限制是我们如何才能提出更好的基准和更好的使用工具的雄心，然后揭示我们现在看到的情报的障碍。

Lenny Rachitsky: That's a good segue to you have a very specific way of thinking about AGI and defining what AGI means.

Lenny Rachitsky：这是一个很好的延续，你有一种非常具体的方式来思考 AGI 并定义 AGI 的含义。

Benjamin Mann: I think AGI is kind of a loaded term, and so I tend not to use it very much anymore internally.

本杰明·曼（Benjamin Mann）：我认为 AGI 是一个含义丰富的术语，所以我倾向于不再在内部使用它。

Instead, I like the term transformative AI because it's less about can it do as much as people do?

相反，我喜欢“变革性人工智能”这个词，因为它不太关心它能否像人类一样做那么多事情？

Can it do literally everything and more about objectively is it causing transformation in society and the economy?

它真的能做到所有事情吗？客观上它是否会导致社会和经济转型？

A very concrete way of measuring that is the Economic Turing Test.

一种非常具体的衡量方法是经济图灵测试。

I didn't come up with this, but I really like it.

我没有想出这个，但我真的很喜欢它。

It's this idea that if you contract an agent for a month or three months on a particular job, if you decide to hire that agent and it turns out to be a machine rather than a person, then it's passed the Economic Turing Test for that role.

这个想法是，如果你与一名代理人签订为期一个月或三个月的特定工作合同，如果你决定雇用该代理人，而结果证明它是一台机器而不是一个人，那么它就通过了该角色的经济图灵测试。

Benjamin Mann: And then you can sort of expand that out in the same way that for measuring purchasing power parity or inflation, there's a basket of goods.

本杰明·曼：然后你可以用与衡量购买力平价或通货膨胀相同的方式将其扩展，有一篮子商品。

You can have a market basket of jobs, and if the agent can pass the Economic Turing Test for 50% of money-weighted jobs, then we have transformative AI and the exact thresholds don't really matter that much, but it's kind of illustrative to say if we pass that threshold, then we would expect massive effects on world GDP increases and societal change and how many people are employed and things like that because societal institutions and organizations are sticky, it's slow to have change, but once these things are possible you know that it's the start of a new era.

你可以拥有一篮子工作，如果代理人能够通过 50% 货币加权工作的经济图灵测试，那么我们就拥有了变革性的人工智能，确切的阈值并不那么重要，但它是一种说明性的说法，如果我们通过了这个阈值，那么我们预计会对世界 GDP 增长和社会变革以及就业人数等产生巨大影响，因为社会机构和组织是粘性的，改变很慢，但一旦这些事情发生了也许你知道这是一个新时代的开始。

Lenny Rachitsky: Along these lines, Dario, your CO recently talked about how AI is going to take a huge part of, I don't know, half of white-collar jobs, that unemployment might go up to something like 20%.

Lenny Rachitsky：按照这个思路，Dario，你的 CO 最近谈到人工智能将如何占据一半白领工作的很大一部分，我不知道，失业率可能会上升到 20% 左右。

I know you're even more vocal and opinionated about just how much impact AI is already having in the workplace that people may not even be realizing.

我知道您对人工智能已经在工作场所产生的影响更加直言不讳和固执己见，而人们甚至可能没有意识到这一点。

Talk about just what you think people are missing about the impact AI is going to have on jobs and is already having.

谈谈你认为人们对人工智能将对工作产生的影响以及已经产生的影响所忽视的部分。

Benjamin Mann: Yeah, so from an economic standpoint, there's a couple different kinds of unemployment, and one is because the workers just don't have the skills to do the kinds of jobs that the economy needs.

本杰明·曼：是的，从经济角度来看，存在几种不同类型的失业，其中一种是因为工人不具备从事经济所需工作的技能。

And another kind is where those jobs are just completely eliminated, and I think it's going to be actually a combination of these things, but if you just think about 20 years in the future where we're way past the singularity, it's hard for me to imagine that even capitalism will look at all it looks today.

另一种是那些工作岗位被完全消除，我认为这实际上是这些事情的结合，但如果你想想未来 20 年，我们已经超越了奇点，我很难想象即使是资本主义也会看到今天的样子。

If we do our jobs, we will have safe aligned superintelligence, we'll have, as Dario says, in Machines of Love and Grace, a country of geniuses in a data center, and the ability to accelerate positive change in science, technology, education, mathematics, it's going to be amazing.

如果我们做好我们的工作，我们将拥有安全一致的超级智能，正如达里奥所说，在爱与恩典的机器中，我们将拥有一个数据中心天才的国家，并且有能力加速科学、技术、教育、数学方面的积极变革，这将是惊人的。

Benjamin Mann: But that also means in a world of abundance where labor is almost free and anything you want to do, you can just ask an expert to do for you, then what do jobs even look like?

本杰明·曼：但这也意味着在一个劳动力几乎免费的丰富世界里，你想做的任何事情，你都可以请专家为你做，那么工作是什么样的呢？

And so I guess there's this scary transition period from where we are today where people have jobs and capitalism works and the world of 20 years from now where everything is completely different, but part of the reason they call it the singularity is that it's a point beyond which you can't easily forecast what's going to happen.

所以我猜想，从今天的情况来看，人们有工作，资本主义在运作，而 20 年后的世界一切都完全不同，但他们称之为奇点的部分原因是，超过这个点你就无法轻易预测将会发生什么。

It's just such a fast rate of change and so different that it's hard to even imagine.

变化的速度是如此之快，而且如此不同，以至于难以想象。

I guess taking the view from the limit, it's pretty easy to say hopefully we'll have figured it out.

我想从极限角度来看，很容易说希望我们能找到答案。

And in a world of abundance, maybe the jobs themselves, it's not that scary, and I think making sure that that transition time goes well is pretty important.

在一个物质丰富的世界里，也许工作本身并没有那么可怕，我认为确保过渡时间顺利非常重要。

Lenny Rachitsky: There's a couple of threads I want to follow there.

Lenny Rachitsky：我想关注几个主题。

One is people hear this, there's a lot of headlines around this.

一是人们听到了这个，有很多关于这个的头条新闻。

Most people probably don't actually feel this yet or see this happening and so there's always this, I guess, I don't know, maybe, but I don't know it's hard to believe, my job seems fine.

大多数人可能还没有真正感受到这一点或看到这种情况发生，所以总是有这种情况，我想，我不知道，也许，但我不知道这很难相信，我的工作看起来很好。

Nothing's changed.

什么都没有改变。

What are you seeing just happening today already that you think people don't see or misunderstand in terms of the impact AI is having on jobs?

对于人工智能对就业的影响，您认为今天刚刚发生的哪些事情人们没有看到或误解？

Benjamin Mann: I think part of this is that people are really bad at modeling exponential progress.

本杰明·曼：我认为部分原因是人们在建模指数级进步方面确实很糟糕。

And if you look at an exponential on a graph, it looks flat and almost zero at the beginning of it, and then suddenly you hit the knee of the curve and things are changing real fast and then it goes vertical.

如果你看一下图表上的指数，它看起来很平坦，一开始几乎为零，然后突然到达曲线的拐点，事情变化得非常快，然后它变得垂直。

That's the plot that we've been on for a long time.

这就是我们长期以来一直在讨论的情节。

I guess I started feeling it in 2019 maybe when GPT-2 came out and I was like, "Oh, this is how we're going to get to AGI." But I think that was pretty early compared to a lot of people where when they saw ChatGPT, they were like, "Wow, something is different and changing." And so I guess I wouldn't expect widespread transformation in a lot of parts of society, and I would expect this skepticism reaction.

我想我大概是在 2019 年 GPT-2 出现的时候开始有这种感觉的，当时我想，“哦，这就是我们实现 AGI 的方式。”但我认为，与很多人看到 ChatGPT 时的感觉相比，这还很早，“哇，有些东西不同了，正在发生变化。”所以我想我不会期望社会的很多部分发生广泛的转变，而且我会期望这种怀疑反应。

I think it's very reasonable and it's exactly what is the standard linear view of progress.

我认为这是非常合理的，这正是标准的线性进步观。

Benjamin Mann: But I guess to cite a couple of areas where I think things are changing quite quickly.

本杰明·曼：但我想举几个我认为事情正在迅速变化的领域。

In customer service we're seeing with things like Fin and Intercom, they're a great partner of ours, 82% customer service resolution rates automatically without a human involved.

在客户服务方面，我们看到 Fin 和 Intercom 等产品，它们是我们出色的合作伙伴，无需人工参与即可自动解决 82% 的客户服务解决率。

And in terms of software engineering, our Claude Code team, like 95% of the code is written by Claude.

而且在软件工程方面，我们Claude Code团队，好像95%的代码都是Claude写的。

But I think a different way to phrase that is that we write 10X more code or 20X more code, and so a much, much smaller team can just be much, much more impactful.

但我认为换一种表达方式是，我们编写了 10 倍或 20 倍的代码，因此规模小得多的团队可能会产生更大的影响力。

And similarly for the customer service, yes, you can phrase it as 82% customer service resolution rates, but that nets out in the humans doing those tasks, able to focus on the harder parts of those tasks.

同样，对于客户服务，是的，您可以将其表述为 82% 的客户服务解决率，但这取决于执行这些任务的人员能够专注于这些任务中更困难的部分。

And for the more tricky situations that in a normal world like five years ago, they would've had to just drop those tickets because it was too much effort for them to actually go do the investigation.

对于更棘手的情况，在五年前的正常世界中，他们将不得不放弃这些罚单，因为对他们来说实际进行调查需要花费太多精力。

There were too many other tickets for them to worry about.

还有太多其他门票让他们担心。

Benjamin Mann: I think in the immediate term, there will be a massive expansion of the pie and the amount of labor that people can do.

本杰明·曼：我认为在短期内，馅饼和人们可以做的劳动量将会大幅扩大。

I've never met a hiring manager at a growth company and heard them say, "I don't want to hire more people." That's the hopeful version of it.

我从来没有见过成长型公司的招聘经理，也没有听过他们说：“我不想雇用更多的人。”这就是它的充满希望的版本。

But with things that are lower skill jobs or less headroom on how good they can be, I think there will be a lot of displacement.

但对于技能较低的工作或它们的发展空间较小，我认为将会有很多人被取代。

It is just something we as a society need to get ahead of and work on.

这正是我们作为一个社会需要超越并努力的事情。

Lenny Rachitsky: Okay.

莱尼·拉奇茨基：好的。

I want to talk more about that, but something that I also want to help people with is how do they get a leg up in this future world?

我想更多地谈论这个问题，但我还想帮助人们的是他们如何在未来的世界中站稳脚跟？

They listen to this, they're like, "Oh, this doesn't sound great.

他们听到这个，他们会说，“哦，这听起来不太好。

I need to think ahead." I know you won't have all the answers, but just do you have any advice for folks that want to try to get ahead of this and kind of future-proof their career and their life to not be replaced by AI?

我需要提前考虑。”我知道你不会得到所有的答案，但对于那些想要尝试超越这一点并确保他们的职业和生活不会被人工智能取代的人，你有什么建议吗？

Anything you've seen people do, anything you recommend they start trying to do more of?

您见过人们做过什么，您建议他们开始尝试做更多的事情吗？

Benjamin Mann: Even for me and being in the center of a lot of this transformation, I'm not immune to job replacement either.

本杰明·曼：即使对我来说，处于许多这种转变的中心，我也不能免受工作替代的影响。

Just some vulnerability there of at some point it's coming for all of us.

只是在某些时候我们所有人都会遇到一些脆弱性。

Lenny Rachitsky: Oh, wait, we've gone too far now.

Lenny Rachitsky：哦，等等，我们现在已经走得太远了。

Okay.

好的。

Benjamin Mann: But in terms of the transition period, yeah, I think there are things that we can do, and I think a big part of it is just being ambitious and how you use the tools and being willing to learn new tools.

本杰明·曼：但就过渡期而言，是的，我认为我们可以做一些事情，我认为其中很大一部分就是雄心勃勃，以及如何使用工具并愿意学习新工具。

People who use the new tools as if they were old tools tend to not succeed.

像使用旧工具一样使用新工具的人往往不会成功。

As an example of that, when you're coding, people are very familiar with autocomplete, people are familiar with SimpleChat where they can ask questions about the code base, but the difference between people who use Claude Code very effectively and people who use it not so effectively is like are they asking for the ambitious change?

举个例子，当你编码时，人们非常熟悉自动完成，人们熟悉 SimpleChat，他们可以在其中询问有关代码库的问题，但是非常有效地使用 Claude Code 的人和使用不太有效的人之间的区别就像他们是否要求雄心勃勃的改变一样？

And if it doesn't work the first time, asking three more times because our success rate when you just completely start over and try again is much, much higher than if you just try once and then just keep banging on the same thing that didn't work.

如果第一次不起作用，请再问三次，因为当你完全重新开始并再次尝试时，我们的成功率比你只尝试一次然后继续尝试同样不起作用的事情要高得多。

Benjamin Mann: And even though that's a coding example and coding is one of the areas that's taking off most dramatically, we have seen internally that our legal team and our finance team are getting a ton of value out of using Claude Code itself.

本杰明·曼：尽管这是一个编码示例，并且编码是发展最迅速的领域之一，但我们在内部看到，我们的法律团队和财务团队正在从使用 Claude Code 本身中获得大量价值。

We're going to be making better interfaces so that they will have an easier time and require a little bit less jumping in the deep end of using Claude Code in the terminal.

我们将制作更好的界面，以便他们能够更轻松地使用 Claude Code，并且在终端中使用 Claude Code 时需要更少的跳跃。

But yeah, we're seeing them use it to redline documents and use it to run BigQuery analyses of our customers and our revenue metrics.

但是，是的，我们看到他们使用它来红线文档，并使用它对我们的客户和收入指标进行 BigQuery 分析。

I guess it's about taking that risk and even if it feels like a scary thing, trying it out.

我想这就是要冒这个风险，即使感觉起来很可怕，也要尝试一下。

Lenny Rachitsky: Okay, so the advice here is use the tools.

Lenny Rachitsky：好的，这里的建议是使用工具。

That's something everyone's always saying, just actually use these tools.

这是每个人总是说的，只要实际使用这些工具即可。

It's like sit in Claude Code.

就像坐在克劳德密码里一样。

And your point about being more ambitious than you naturally feel like being because maybe it'll actually accomplish the thing.

你的观点是比你自然想象的更加雄心勃勃，因为也许它真的能实现目标。

This tip of trying it three times so the idea there is it may not get it right the first time.

这个技巧需要尝试三次，所以第一次可能不会成功。

Is the tip there ask it in different ways or is it just try harder, try again?

那里的提示是否以不同的方式询问它，或者只是更加努力，再试一次？

Benjamin Mann: Yeah, I mean you can just literally ask the exact same question.

本杰明·曼：是的，我的意思是你可以直接问完全相同的问题。

These things are stochastic and sometimes they'll figure it out and sometimes they won't.

这些事情是随机的，有时他们会弄清楚，有时却不会。

In every one of these model cards, it always shows pass it one versus pass it in.

在每一张模型卡中，它总是显示“pass it one”与“pass it in”。

And that's exactly the thing where they try the exact same prompt, sometimes it gets it, sometimes it doesn't.

这正是他们尝试完全相同的提示的情况，有时会得到，有时则不会。

That's the dumbest advice.

这是最愚蠢的建议。

But yeah, I think if you want to be a little bit smarter about it, there can be gains there of saying, "Here's what you already tried and it didn't work, so don't try that.

但是，是的，我认为如果你想在这方面变得更聪明一点，可以说：“这是你已经尝试过的方法，但它不起作用，所以不要尝试这样做。

Try something different." That can also help.

尝试一些不同的东西。”这也有帮助。

Lenny Rachitsky: The advice is comes back to something that a lot of people talk about these days is you won't be replaced by AI at least anytime soon you'll be replaced by someone that is very good using AI?

Lenny Rachitsky：这个建议又回到了现在很多人谈论的事情，那就是你不会被人工智能取代，至少在短期内你会被非常擅长使用人工智能的人取代？

Benjamin Mann: I think in that area it's more like your team will just do dramatically more stuff.

本杰明·曼：我认为在这个领域，你的团队会做更多的事情。

We're definitely not slowing down on hiring at all, and some people are confused by that.

我们绝对不会放慢招聘速度，有些人对此感到困惑。

Even in an onboarding class, somebody asked that and they were like, "Why did you hire me if we're all just going to be replaced?" And the answer is the next couple of years are really critical to get right and we're not at the point where we're doing complete replacement.

即使在入职培训班上，也会有人问这个问题，他们会说：“如果我们都将被替换，那你为什么还要雇用我呢？”答案是，未来几年对于正确实施非常关键，而且我们还没有到完全更换的地步。

Like I said, we're still at that flat zero looking part of the exponential compared to where we will be.

就像我说的，与我们将要达到的位置相比，我们仍然处于指数的平坦零部分。

It is super important to have great people and that's why we're hiring super aggressively.

拥有优秀的人才非常重要，这就是我们如此积极招聘的原因。

Lenny Rachitsky: Let me take another approach to asking this question something ask everyone that's at the very cutting edge of where AI is going.

Lenny Rachitsky：让我用另一种方法来问这个问题，问每个处于人工智能发展最前沿的人。

You have kids, knowing what you know about where AI is heading and all these things you've been talking about, what are you focusing on teaching your kids to help them thrive in this AI future?

你有孩子，知道人工智能的发展方向以及你一直在谈论的所有这些事情，你会重点教你的孩子什么来帮助他们在人工智能的未来中茁壮成长？

Benjamin Mann: Yeah, I have two daughters, a one-year-old and a three-year-old, so it's pretty in the basics still.

本杰明·曼：是的，我有两个女儿，一个一岁，一个三岁，所以基本来说还是很漂亮的。

And our three-year-old is now capable of just conversing with Alexa Plus and asking her to explain stuff and play music for her and all that stuff.

我们三岁的孩子现在能够与 Alexa Plus 交谈并要求她解释东西并为她播放音乐等等。

She's been loving that.

她一直很喜欢这个。

But I guess more broadly, she goes to a Montessori school and I just love the focus on curiosity and creativity and self-led learning that Montessori has.

但我想更广泛地说，她去了蒙特梭利学校，我只是喜欢蒙特梭利对好奇心、创造力和自我引导学习的关注。

Benjamin Mann: I guess if I were in a normal era like 10, 20 years ago and I had a kid, maybe I would be trying to line her up for going to a top tier school and doing all the extracurriculars and all that stuff.

本杰明·曼：我想如果我在一个正常的时代，比如 10、20 年前，我有一个孩子，也许我会努力让她去一所顶级学校，参加所有的课外活动等等。

But at this point, I don't think any of it's going to matter.

但在这一点上，我认为这一切都不重要了。

I just want her to be happy and thoughtful and curious and kind.

我只希望她快乐、体贴、好奇、善良。

And the Montessori school is definitely doing great at that.

蒙特梭利学校在这方面绝对做得很好。

They text us throughout the day.

他们全天给我们发短信。

Sometimes they're like, "Oh, your kid got in an argument with this other kid and she has really big emotions and she tried to use her words." I love that.

有时他们会说，“哦，你的孩子和另一个孩子发生了争执，她的情绪非常激动，她试图用自己的话来表达。”我喜欢那个。

I think that's exactly the kind of education that I think is most important, that the facts are going to fade into the background.

我认为这正是我认为最重要的教育，事实将逐渐淡出背景。

Lenny Rachitsky: I'm a huge fan of Montessori also.

Lenny Rachitsky：我也是蒙特梭利的忠实粉丝。

I'm trying to get our kid into Montessori school.

我正在努力让我们的孩子进入蒙特梭利学校。

He's two years old, so we're on the same track.

他两岁了，所以我们在同一个轨道上。

This idea of curiosity, it comes up every single time.

这种好奇的想法每次都会出现。

Ask someone that's working at the cutting edge of AI, what skill to instill in your child and curiosity comes up the most.

问问那些在人工智能前沿工作的人，应该向你的孩子灌输什么技能，什么技能最能激发孩子的好奇心。

I think that's a really interesting takeaway.

我认为这是一个非常有趣的收获。

I think this point about being kind is also really important, especially with our AI overlords trying to be kind to them.

我认为友善这一点也非常重要，尤其是当我们的人工智能霸主试图善待他们时。

I love how people are always saying thank you to Claude.

我喜欢人们总是对克劳德说谢谢。

And then creativity.

然后是创造力。

That's interesting.

那很有意思。

That doesn't come up as much just being creative.

仅仅靠创意并不能解决这个问题。

Lenny Rachitsky: I want to go in a different direction.

Lenny Rachitsky：我想朝不同的方向发展。

I want to go back to the beginning of Anthropic.

我想回到《人择》的开头。

Famously you and eight of you left OpenAI back in the day in 2020, I believe the end of 2020 to start Anthropic.

众所周知，您和你们八个人在 2020 年离开了 OpenAI，我相信在 2020 年底会创办 Anthropic。

Talk a little bit about why this happened, what you guys saw.

谈谈为什么会发生这种情况，以及你们看到了什么。

I'm curious, just if you're willing to share more, just what is it that you saw at OpenAI, what'd you experience there that made you feel like, okay, we got to go do our own thing?

我很好奇，如果你愿意分享更多，你在 OpenAI 看到了什么，你在那里经历了什么，让你觉得，好吧，我们必须去做自己的事情？

Benjamin Mann: Yeah, so for the listeners, I was part of the GPT-2=3 project at OpenAI, ended up being one of the first authors on the paper, and I also did a bunch of demos for Microsoft to help raise $1 billion from them, did the tech transfer of GPT-3 to their systems so that they could help serve the model in Azure.

Benjamin Mann：是的，所以对于听众来说，我是 OpenAI 的 GPT-2=3 项目的一部分，最终成为该论文的第一作者之一，我还为微软做了很多演示，以帮助从他们那里筹集 10 亿美元，将 GPT-3 的技术转移到他们的系统中，以便他们可以帮助在 Azure 中提供模型服务。

I did a bunch of different things there on both the more researchy side and the product side.

我在研究方面和产品方面做了很多不同的事情。

One weird thing about OpenAI is that while I was there, Sam talked about having three tribes that needed to be kept in check with each other, which was the safety tribe, the research tribe, and the startup tribe.

关于 OpenAI 的一件奇怪的事情是，当我在那里时，Sam 谈到了三个需要相互控制的部落，即安全部落、研究部落和创业部落。

And whenever I heard that, it just struck me as the wrong way to approach things because the company's mission apparently is to make the transition to AGI safe and beneficial for humanity.

每当我听到这个消息时，我都会觉得这是一种错误的处理方式，因为该公司的使命显然是让向 AGI 的过渡安全且造福于人类。

Benjamin Mann: And that's basically the same as Anthropic's mission.

本杰明·曼：这与 Anthropic 的使命基本相同。

But internally, it felt like there was so much tension around these things.

但在内心深处，这些事情让人感到非常紧张。

And I think when push came to shove, we felt like safety wasn't the top priority there.

我认为，当事情到了紧要关头时，我们觉得安全并不是首要任务。

And there are good reasons that you might think that if you thought safety was going to be easy to solve or if you thought it wasn't going to have a big impact, or if you thought that the chance of big negative outcomes was vanishingly small, then maybe you would just do those kinds of actions.

你可能有充分的理由认为，如果你认为安全问题很容易解决，或者你认为它不会产生很大的影响，或者你认为出现重大负面结果的可能性微乎其微，那么也许你就会采取这些行动。

But at Anthropic we felt, I mean we didn't exist then, but it was basically the leads of all the safety teams at OpenAI, we felt that safety is really important, especially on the margin.

但在 Anthropic，我们觉得，我的意思是我们当时还不存在，但基本上是 OpenAI 所有安全团队的领导，我们觉得安全非常重要，尤其是在边缘。

And so if you look at who in the world is actually working on safety problems, it's pretty small set of people.

因此，如果你看看世界上真正致力于解决安全问题的人，就会发现只有一小部分人。

Even now, I mean the industry is blowing up, as I mentioned, 300 billion a year CapEx today, and I would say maybe less than 1,000 people working on it worldwide, which is just crazy.

即使是现在，我的意思是，正如我提到的，这个行业正在蓬勃发展，如今每年的资本支出达 3000 亿美元，而我想说，全世界从事该行业的人员可能还不到 1000 人，这太疯狂了。

Benjamin Mann: That was fundamentally why we left.

本杰明·曼：这就是我们离开的根本原因。

We felt like we wanted an organization where we could be on the frontier, we could be doing the fundamental research, but we could be prioritizing safety ahead of everything else.

我们觉得我们想要一个可以处于前沿的组织，我们可以进行基础研究，但我们可以将安全放在首位。

And I think that's really panned for us in a surprising way.

我认为这对我们来说确实是令人惊讶的。

We didn't know even if it would be possible to make progress on the safety research because at the time, we had tried a bunch of safety through debate and the models weren't good enough.

我们甚至不知道安全性研究是否有可能取得进展，因为当时我们通过辩论尝试了很多安全性，但模型还不够好。

And so we basically had no results on all of that work, and now that exact technique is working and many others that we have been thinking about for a long time.

所以我们基本上所有这些工作都没有结果，现在精确的技术正在发挥作用，还有我们长期以来一直在考虑的许多其他技术。

Yeah, fundamentally it comes down to is safety the number one priority?

是的，从根本上来说，安全是第一位的吗？

And then something that we've sort of tacked on since then is like, can you have safety and be at the front here at the same time?

从那时起，我们就一直在强调这样一个问题：你能保证安全，同时站在前面吗？

Benjamin Mann: And if you look at something like sycophancy, I think Claude is one of the least sycophantic models because we've put so much effort into actual alignment and not just trying to good heart our metrics of saying user engagement is number one, and if people say yes, then it's good for them.

本杰明·曼：如果你看看阿谀奉承之类的东西，我认为克劳德是最不阿谀奉承的模型之一，因为我们在实际协调上投入了很多精力，而不仅仅是试图用我们的指标来表示用户参与度是第一，如果人们说是，那么这对他们有好处。

Lenny Rachitsky: Okay.

莱尼·拉奇茨基：好的。

Let's talk about this tension that you mentioned, this tension between safety and progress, being competitive in the marketplace.

让我们谈谈你提到的这种紧张关系，这种安全与进步、市场竞争之间的紧张关系。

I know you spent a lot of your time on safety.

我知道您在安全方面花了很多时间。

I know that as you just alluded to, this is a core part of how you think about AI.

我知道，正如您刚才提到的，这是您对人工智能的看法的核心部分。

I want to talk about why that is, but first of all, just how do you think about this tension between focusing on safety while also not falling way behind?

我想谈谈为什么会这样，但首先，您如何看待关注安全与不落后之间的这种紧张关系？

Benjamin Mann: Yeah, so initially we thought that it would be sort of one or the other, but I think since then we've realized that it's actually kind of convex in the sense that working on one helps us with the other thing.

本杰明·曼：是的，所以最初我们认为这会是其中之一，但我认为从那时起我们已经意识到它实际上是凸的，因为致力于一件事可以帮助我们解决另一件事。

Initially when Opus 3 came out and we were finally at the frontier of model capabilities, one of the things that people really loved about it was the character and the personality.

最初，当 Opus 3 推出时，我们终于处于模型功能的前沿，人们真正喜欢它的事情之一就是它的角色和个性。

And that was directly a result of our alignment research.

这是我们对齐研究的直接结果。

Amanda Askell did a ton of work on this and as well as many others who tried to figure out what does it mean for an agent to be helpful, honest, and heartless, and what does it mean to be in difficult conversations and show up effectively?

阿曼达·阿斯克尔（Amanda Askell）在这方面做了大量的工作，还有许多其他人试图弄清楚代理人乐于助人、诚实和无情意味着什么，以及在困难的对话中有效地出现意味着什么？

How do you do a refusal that doesn't shut the person down, but makes them feel like they understand why the agent said, "I can't help you with that.

你如何拒绝，既不会让对方失望，又能让他们觉得自己理解代理人为什么说：“我帮不了你。

Maybe you should talk to a medical professional, or maybe you should consider not trying to build bio-weapons or something like that."

也许你应该咨询医疗专家，或者你应该考虑不要尝试制造生物武器或类似的东西。”

Benjamin Mann: Yeah, I guess that's part of it.

本杰明·曼：是的，我想这就是其中的一部分。

And then another piece that's come out is constitutional ai, where we have this list of natural language principles that leads the model to learn how we think a model should behave.

然后出现的另一件作品是宪法人工智能，其中我们有一系列自然语言原则，可以引导模型学习我们认为模型应该如何表现。

And they've been taken from things like the UN Declaration of Human Rights and Apple's privacy terms of service and a whole bunch of other places, many of which we've just generated ourselves that allow us to take a more principled stance, not just leaving it to whatever human raiders we happen to find, but we ourselves deciding what should the values of this agent be?

它们取自《联合国人权宣言》、苹果的隐私服务条款和一大堆其他地方，其中许多是我们自己生成的，使我们能够采取更有原则的立场，而不是仅仅将其留给我们碰巧发现的任何人类袭击者，而是我们自己决定该代理的价值观应该是什么？

And that's been really valuable for our customers because they can just look at that list and say like, "Yep, these seem right.

这对我们的客户来说非常有价值，因为他们只需查看该列表即可说：“是的，这些看起来是正确的。

I like this company, I like this model.

我喜欢这个公司，我喜欢这个模式。

I trust it."

我相信它。”

Lenny Rachitsky: Okay, this is awesome.

Lenny Rachitsky：好的，这太棒了。

One nugget there is your point that the personality of Claude, its personality is directly aligned with safety.

你的一个要点是克劳德的个性，它的个性与安全直接相关。

I don't think a lot of people think about that.

我认为很多人都没有考虑过这一点。

And this is because of the values that you imbue, is that the word, with constitutional AI and things like that.

这是因为你灌输的价值观，就是这个词，宪法人工智能之类的东西。

Like the actual personality of the AIs directly connected to your focus on safety.

就像人工智能的实际个性与您对安全的关注直接相关。

Benjamin Mann: That's right.

本杰明·曼：没错。

That's right.

这是正确的。

And from a distance, it might seem quite disconnected, like how is this going to prevent X risk?

从远处看，它可能看起来完全无关，比如这将如何防止 X 风险？

But ultimately it's about the AI understanding what people want and not what they say.

但归根结底，人工智能要理解人们想要什么，而不是他们说什么。

We don't want the Monkey Paw Scenario of the genie gives these three wishes and then you end up having everything you touch turns of gold.

我们不希望精灵的猴爪场景满足这三个愿望，然后你所接触的一切都会变成金色。

We want the AI to be like, oh, obviously what you really meant was this, and that's what I'm going to help you with.

我们希望人工智能能够像，哦，显然你真正的意思是这个，这就是我要帮助你的。

I think it is really quite connected.

我认为这确实是有很大联系的。

Lenny Rachitsky: Talk a bit more about this constitutionally AI.

Lenny Rachitsky：多谈谈这个本质上的人工智能。

This is essentially you bake in, here's the rules that we want you to abide by and it's values, you said it's the Geneva Human Rights Code, things like that.

这本质上是你融入的，这是我们希望你遵守的规则及其价值观，你说这是《日内瓦人权法典》，诸如此类的东西。

How does that actually work?

这实际上是如何运作的？

I think the core here is just this is baked into the model.

我认为这里的核心就是将其融入到模型中。

It's not something you add on top later.

这不是你以后添加的东西。

Benjamin Mann: I'll just give a quick overview of how constitutionally AI actually works.

本杰明·曼（Benjamin Mann）：我将简要概述一下人工智能的实际运作方式。

Benjamin Mann: The idea is the model is going to produce some output with some input by default before we've done our safety and helpful and harmlessness training.

本杰明·曼（Benjamin Mann）：我们的想法是，在我们完成安全、有用和无害的训练之前，模型将默认产生一些输出和一些输入。

Let's say an example is write me a story, and then the constitutional principles might include things like people should be nice to each other and not have hate speech, and you should not expose somebody's credentials if they give them to you in a trusting relationship.

举个例子，给我写一个故事，然后宪法原则可能包括人们应该彼此友善而不发表仇恨言论，如果某人以信任关系向您提供凭据，您不应该暴露他们的凭据。

And so some of these constitutional principles might be more or less applicable to the prompt that was given.

因此，其中一些宪法原则可能或多或少适用于所给出的提示。

And so first we have to figure out which ones might apply.

因此，首先我们必须弄清楚哪些可能适用。

And then once we figure that out, then we ask the model itself to first generate a response and then see does the response actually abide by the constitutional principle?

那么一旦我们弄清楚了这一点，那么我们就要求模型本身首先生成一个响应，然后看看这个响应是否真的符合宪法原则？

And if the answer is, yep, I was great, then nothing happens.

如果答案是，是的，我很棒，那么什么也不会发生。

But if the answer is no, actually I wasn't in compliance with the principle, then we ask the model itself to critique itself and rewrite its own response in light of the principle, and then we just remove the middle part where it did the extra work.

但如果答案是否定的，实际上我没有遵守原则，那么我们要求模型本身进行自我批判，并根据原则重写自己的响应，然后我们删除中间它做了额外工作的部分。

Benjamin Mann: And then we say, "Okay, in the future just produce the correct response out the gate." And that simple process, hopefully it sounded simple.

本杰明·曼：然后我们说，“好吧，将来只要给出正确的反应就可以了。”这个简单的过程，希望听起来很简单。

Benjamin Mann: It is just using the model to improve itself recursively and align itself with these values that we've decided are good.

本杰明·曼（Benjamin Mann）：它只是使用模型来递归地改进自身，并使自身与我们认为好的价值观保持一致。

And this is also not something that we think as a small group of people in San Francisco should be figuring out.

我们也不认为旧金山的一小群人应该弄清楚这一点。

This should be a society wide conversation.

这应该是全社会的对话。

And that's why we've published the Constitution.

这就是我们出版宪法的原因。

And we've also done a bunch of research on defining a collective constitution where we ask a lot of people what their values are and what they think an AI model should behave like.

我们还做了很多关于定义集体宪法的研究，我们询问了很多人他们的价值观是什么，以及他们认为人工智能模型应该是什么样子。

But yeah, this is all an ongoing area of research where we're constantly iterating.

但是，是的，这都是我们不断迭代的持续研究领域。

Lenny Rachitsky: This episode is brought to you by Fin, the number one AI agent for customer service.

Lenny Rachitsky：本集由 Fin 为您带来，Fin 是第一大客户服务人工智能代理。

If your customer support tickets are piling up, then you need Finn.

如果您的客户支持请求堆积如山，那么您需要 Finn。

Fin.

鳍。

Fin is the highest performing AI agent on the market with a 59% average resolution rate.

Fin 是市场上性能最高的 AI 代理，平均分辨率为 59%。

Fin resolves even the most complex customer queries.

Fin 甚至可以解决最复杂的客户查询。

No other AI agent performs better.

没有其他人工智能代理表现得更好。

In head head bake-offs with competitors.

与竞争对手进行头头比拼。

Fin wins every time.

芬每次都获胜。

Yes, switching to a new tool can be scary, but Fin works on any help desk with no migration needed, which means you don't have to overhaul your current system or deal with delays in service for your customers.

是的，切换到新工具可能会令人恐惧，但 Fin 可以在任何帮助台上工作，无需迁移，这意味着您不必彻底检修当前系统或处理客户服务延迟的问题。

And Fin is trusted by over 5,000 customer service leaders and top AI companies like Anthropic and Synthesia.

Fin 受到超过 5,000 家客户服务领导者以及 Anthropic 和 Synthesia 等顶级人工智能公司的信赖。

And because Fin is powered by the Fin AI engine, which is a continuously improving system that allows you to analyze, train, test, and deploy with ease, Fin can continuously improve your results too.

由于 Fin 由 Fin AI 引擎提供支持，这是一个不断改进的系统，可让您轻松分析、训练、测试和部署，因此 Fin 也可以不断改进您的结果。

Lenny Rachitsky: If you're ready to transform your customer service and scale your support, give Finn a try for only .99 cents per resolution.

Lenny Rachitsky：如果您准备好转变您的客户服务并扩大您的支持范围，请尝试一下 Finn，每个解决方案仅需 0.99 美分。

Plus Fin comes with a 90-day money back guarantee.

Plus Fin 提供 90 天退款保证。

Find out how Finn can work for your team at fin.ai/lenny.

请访问 fin.ai/lenny，了解 Finn 如何为您的团队工作。

That's fin.ai/lenny.

那是 fin.ai/lenny。

Lenny Rachitsky: I'm going to kind of zoom out a little bit and talk about just why this is so core to you.

Lenny Rachitsky：我要稍微缩小一下范围，谈谈为什么这对你来说如此重要。

What was your inception of just like, holy shit, I need to focus on this with everything I do in ai?

你是怎么想到的，天哪，我在人工智能领域所做的一切都需要专注于这个？

Obviously it became a central part of Anthropic's mission more than any other company.

显然，它比任何其他公司都更能成为 Anthropic 使命的核心部分。

A lot of people talk about safety, like you said, only maybe 1,000 people actually work on it.

很多人都在谈论安全，就像你说的，实际上只有 1000 人在做这方面的工作。

I feel like you're at the top of that pyramid of actually having the impact on this.

我觉得你处于金字塔的顶端，能够真正对此产生影响。

Why is this so important?

为什么这如此重要？

What do you think people maybe are missing or don't understand?

您认为人们可能遗漏或不理解什么？

Benjamin Mann: For me, I read a lot of science fiction growing up, and I think that sort of positioned me to think about things in a long-term view.

本杰明·曼：对我来说，我在成长过程中读了很多科幻小说，我认为这让我能够从长远的角度思考事物。

And a lot of science fiction books are like space operas where humanity is a multi galactic civilization has extremely advanced technology building Dyson spheres around the sun with sentient robots to help them.

许多科幻小说就像太空歌剧，人类是一个多星系文明，拥有极其先进的技术，可以在太阳周围建造戴森球，并有有感知的机器人来帮助它们。

And so for me, coming from that world, it wasn't like a huge leap to imagine machines that could think.

因此，对于来自那个世界的我来说，想象能够思考的机器并不是一个巨大的飞跃。

But when I read Superintelligence by Nick Bostrom in around 2016, it really became real for me where he just describes how hard it will be to make sure that an AI system trained with the kinds of optimization techniques that we had at the time would be anywhere near aligned, would even understand our values at all.

但当我在 2016 年左右读到尼克·博斯特罗姆 (Nick Bostrom) 写的《超级智能》时，这对我来说真的变得真实起来，他只是描述了要确保一个经过我们当时拥有的各种优化技术训练的人工智能系统几乎一致，甚至完全理解我们的价值观是多么困难。

And since then, my estimation of how hard the problem would be has gone down significantly actually, because things like language models actually do really understand human values in a core way.

从那时起，我对问题难度的估计实际上已经大大下降，因为像语言模型这样的东西实际上确实以核心方式真正理解人类价值观。

Benjamin Mann: The problem is definitely not solved, but I'm more hopeful than I was.

本杰明·曼：问题肯定没有解决，但我比以前更有希望了。

But since I read that book, I immediately decided I had to join OpenAI, so I did.

但自从读了那本书后，我立即决定必须加入 OpenAI，所以我就这么做了。

And at the time, there were a tiny research lab with basically no claim to fame at all.

当时，有一个很小的研究实验室，基本上没有什么名气。

I only knew about them because my friend knew Greg Brockman, who was the CTO at the time.

我之所以知道他们，是因为我的朋友认识格雷格·布罗克曼（Greg Brockman），他当时是首席技术官。

And Elon was there and Sam wasn't really there.

埃隆在那里，而山姆并不在那里。

And it was a very different organization.

这是一个非常不同的组织。

But over time, I think the case for safety has gotten a lot more concrete where when we started OpenAI, it was not clear how we get to AGI.

但随着时间的推移，我认为安全问题已经变得更加具体，当我们开始 OpenAI 时，并不清楚我们如何实现 AGI。

And we were like, maybe we'll need a bunch of RL agents battling it out on a desert island and consciousness will somehow emerge.

我们当时想，也许我们需要一群强化学习特工在荒岛上奋战，意识就会以某种方式出现。

But since then, since language modeling has started working, I think the path has become pretty clear.

但从那时起，由于语言建模开始发挥作用，我认为道路已经变得非常清晰。

Benjamin Mann: I guess now the way I think about the challenges are pretty different from how they're laid out in superintelligence.

本杰明·曼（Benjamin Mann）：我想现在我对挑战的思考方式与超级智能中的挑战方式有很大不同。

Superintelligence is a lot about how do we keep God in a box and not let the God out.

超级智能主要是关于我们如何将上帝关在盒子里而不让上帝出去。

And with language models, it's been kind of both hilarious and terrifying at the same time to see people pulling the God out of the box and being like, "Yeah, come use the whole internet.

有了语言模型，看到人们把上帝从盒子里拉出来，然后说：“是的，来使用整个互联网吧，这既搞笑又可怕。

Here's my bank account, do all sorts of crazy stuff." Just such a different tone from superintelligence.

这是我的银行账户，可以做各种疯狂的事情。”与超级智能的语气完全不同。

And to be clear, I don't think it's actually that dangerous right now.

需要明确的是，我认为现在实际上并没有那么危险。

Our responsible scaling policy defines these AI safety levels that tries to figure out for each level of model intelligence, what is the risk to society.

我们负责任的扩展政策定义了这些人工智能安全级别，试图找出模型智能的每个级别对社会的风险是什么。

And currently we think we're at ASL-3, which is maybe a little bit risk of harm but not significant.

目前我们认为我们处于 ASL-3 级，这可能有一点伤害风险，但并不严重。

Benjamin Mann: ASL-4 starts to get to significant loss of human life if a bad actor misuse the technology.

本杰明·曼：如果坏人滥用 ASL-4 技术，就会造成重大人员伤亡。

And then ASL-5 is potentially extinction level if it's misused or if it is misaligned and does its own thing.

如果 ASL-5 被滥用或未对准并自行其事，则 ASL-5 可能会达到灭绝级别。

We've testified to Congress about how models can do biological uplift in terms of making new pandemics using the models, and that's the A/B test against Google Search.

我们已经向国会证明了模型如何在使用模型制造新的流行病方面实现生物提升，这就是针对 Google 搜索的 A/B 测试。

That's like the previous state of the art on uplift trials.

这就像之前提升试验的最新技术一样。

And we found that with ASL-3 models, it is actually somewhat significant.

我们发现对于 ASL-3 模型来说，这实际上是有一定意义的。

It does really help if you wanted to create a bioweapon, and we've hired some experts who actually how to evaluate for those things, but compared to the future, it's not really anything.

如果你想创造一种生物武器，它确实很有帮助，我们已经聘请了一些专家，他们实际上如何评估这些东西，但与未来相比，这实际上不算什么。

And I think that's another part of our mission of creating that awareness of saying, "If it is possible to do these bad things, then legislators should know what the risks are." And I think that's part of why we're so trusted in Washington because we've been sort of upfront and clear-eyed about what's going on, what's probably going to happen.

我认为这是我们使命的另一部分，即建立这样一种意识：“如果有可能做这些坏事，那么立法者应该知道风险是什么。”我认为这就是为什么我们在华盛顿如此受信任的部分原因，因为我们对正在发生的事情以及可能会发生的事情一直保持着坦率和清醒的态度。

Lenny Rachitsky: It's interesting because you guys put out more examples of your models doing bad things than anyone else.

Lenny Rachitsky：这很有趣，因为你们提出的模型做坏事的例子比其他任何人都多。

There was I think a story of an agent or a model trying to blackmail engineer.

我认为有一个经纪人或模特试图勒索工程师的故事。

You guys had the store that you ran internally that was selling you things and ended up not working out great as losing a lot of money, ordered all these tungsten cubes or something.

你们内部经营的商店向你们出售东西，但结果并不好，因为损失了很多钱，订购了所有这些钨立方体或其他东西。

Is part of that just making sure people are aware of what is possible, just it makes you look bad, right?

其中一部分只是确保人们意识到什么是可能的，只是这会让你看起来很糟糕，对吗？

It's like, oh, our model's messing up in all these different ways.

就像，哦，我们的模型在所有这些不同的方面都搞砸了。

What's the thinking of just sharing all the stories that other companies don't?

仅仅分享其他公司不分享的所有故事是怎么想的？

Benjamin Mann: Yeah, I mean I think there's a traditional mindset where it makes us look bad, but I think if you talk to policymakers, they really appreciate this kind of thing because they feel like we're giving them the straight talk and that's what we strive to do, that they can trust us, that we're not going to paper things over or sugarcoat things.

本杰明·曼：是的，我的意思是，我认为传统的思维方式让我们看起来很糟糕，但我认为如果你与政策制定者交谈，他们真的很欣赏这种事情，因为他们觉得我们正在给他们直言不讳，这就是我们努力做的事情，他们可以信任我们，我们不会掩盖或粉饰事情。

That's been really encouraging.

这确实令人鼓舞。

Yeah, I think for the blackmail thing, it blew up in the news in a weird way where people were like, "Oh, Claude's going to blackmail you in a real life scenario." But it was a very specific laboratory setting that this kind of thing gets investigated in.

是的，我认为对于勒索事件，它以一种奇怪的方式在新闻中爆炸，人们会说，“哦，克劳德会在现实生活中勒索你。”但这种事情的研究是在一个非常特殊的实验室环境中进行的。

And I think that's generally our take of let's have the best models so that we can exercise them in laboratory settings where it's safe and understand what the actual risks are, rather than trying to turn a blind eye and say, "Well, it'll probably be fine." And then let the bad thing happen in the wild.

我认为这通常是我们的看法，让我们拥有最好的模型，这样我们就可以在安全的实验室环境中运用它们，并了解实际的风险是什么，而不是试图视而不见并说：“好吧，可能会没事的。”然后让坏事在野外发生。

Lenny Rachitsky: One of the criticisms you guys get is that you do this to kind of differentiate or raise money to create headlines.

Lenny Rachitsky：你们收到的批评之一是你们这样做是为了差异化或筹集资金来制造头条新闻。

It's like, oh, they're just over there dooming glooming us about where the future is heading.

就像，哦，他们就在那里，注定让我们对未来的走向感到沮丧。

On the other hand, Mike Krieger was on the podcast and he shared how every prediction Dario's had about the progress AI is going to have is just spot on year after year and he's predicting 2027, 28 AGI, something like that so these things start to get real.

另一方面，迈克·克里格 (Mike Krieger) 在播客中分享了达里奥 (Dario) 对人工智能进步的每一个预测都是如何年复一年地准确无误的，他预测 2027 年，AGI 28，类似的事情，所以这些事情开始成为现实。

I guess, what's your response to folks that are just like, "Ah, these guys are just trying to scare us all just to get attention?"

我想，对于那些“啊，这些家伙只是想吓唬我们所有人，只是为了引起注意”的人，你会有什么反应？

Benjamin Mann: I mean, I think part of why we publish these things is we want other labs to be aware of the risks.

本杰明·曼：我的意思是，我认为我们发布这些内容的部分原因是我们希望其他实验室意识到这些风险。

And yes, there could be a narrative of we're doing it for attention, but honestly from a attention grabbing thing, I think there is a lot of other stuff we could be doing that would be more attention grabbing if we didn't actually care about safety.

是的，可能会有这样的说法，我们这样做是为了引起注意，但老实说，从吸引注意力的事情来看，我认为如果我们实际上并不关心安全，我们可以做很多其他事情，这些事情会更吸引注意力。

A tiny example of this is we published a computer using agent reference implementation in our API only because when we built a prototype of a consumer application for this, we couldn't figure out how to meet the safety bar that we felt was needed for people to trust it and for it not to do bad things.

一个小例子是，我们在 API 中使用代理参考实现发布了一台计算机，只是因为当我们为此构建消费者应用程序的原型时，我们无法弄清楚如何满足我们认为需要人们信任它并且不做坏事的安全标准。

And there are definitely safe ways to use the API version that we're seeing a lot of companies use for automated software testing, for example, in a safe way.

肯定有安全的方法来使用我们看到许多公司用于自动化软件测试的 API 版本，例如，以安全的方式。

Benjamin Mann: We could have gone out and hyped that up and said, "Oh my God, Claude can use your computer and everybody should do this today." But we were like, "It's just not ready and we're going to hold it back till it's ready." I think from a hype standpoint, our actions show otherwise.

本杰明·曼：我们本可以出去宣传并说：“天哪，克劳德可以使用你的计算机，今天每个人都应该这样做。”但我们当时想，“它还没有准备好，我们将推迟它，直到它准备好。”我认为从炒作的角度来看，我们的行动却表明情况并非如此。

From a Doomer perspective, it's a good question.

从杜默的角度来看，这是一个很好的问题。

I think my personal feeling about this is that things are overwhelmingly likely to go well, but on the margin almost nobody is looking at the downside risk.

我认为我个人对此的感觉是，事情极有可能进展顺利，但在边缘几乎没有人关注下行风险。

And the downside risk is very large.

而且下行风险非常大。

Once we get to superintelligence, it will be too late to align the models probably.

一旦我们达到超级智能，调整模型可能就为时已晚了。

This is a problem that's potentially extremely hard and that we need to be working on way ahead of time.

这是一个可能非常困难的问题，我们需要提前解决。

And so that's why we're focusing on it so much now.

这就是为什么我们现在如此关注它。

Benjamin Mann: And even if there's only a small chance that things go wrong, to make an analogy, if I told you that there is a 1% chance that the next time you got in an airplane you would die, you probably think twice even though it's only 1% because it's just such a bad outcome.

本杰明·曼：即使出问题的可能性很小，打个比方，如果我告诉你，下次你坐飞机时有 1% 的可能性会死，你可能会三思而后行，尽管只有 1%，因为这是一个非常糟糕的结果。

And if we're talking about the whole future of humanity, it's just a dramatic future to be gambling with.

如果我们谈论的是人类的整个未来，那么这只是一个值得赌博的戏剧性未来。

I think it's more on the sense of yes, things will probably go well, yes, we want to create safe AGI and deliver the benefits to humanity, but let's make triple sure that it's going to go well.

我认为更多的意义是，是的，事情可能会进展顺利，是的，我们希望创建安全的通用人工智能并为人类带来好处，但让我们三重确保它会顺利进行。

Lenny Rachitsky: You wrote somewhere that creating powerful AI might be the last invention humanity ever needs to make.

Lenny Rachitsky：你在某处写道，创造强大的人工智能可能是人类需要做出的最后一项发明。

If it goes poorly, it can mean a bad outcome for humanity forever.

如果进展不佳，可能会给人类带来永远的糟糕结果。

If it goes well, the sooner it goes well, the better.

如果顺利的话，越早顺利越好。

Such a beautiful way to summarize it.

这是一个很好的总结方式。

We had a recent guest, Sandra Schulhoff, who pointed out that AI right now it's like just on a computer, you could maybe search just the web, but there's only so much harm it could do.

我们最近邀请了一位嘉宾桑德拉·舒尔霍夫 (Sandra Schulhoff)，她指出人工智能现在就像在计算机上一样，你可能只能搜索网络，但它能造成的伤害也有限。

But when it starts to go into robots and all these autonomous agents, that's when it really starts, like physically becomes dangerous if we don't get this right.

但当它开始进入机器人和所有这些自主代理时，那就是它真正开始的时候，如果我们不做对的话，身体就会变得危险。

Benjamin Mann: Yeah, I think there's some nuance to that where if you look at how North Korea makes a significant fraction of its economy revenue, it's from hacking crypto exchanges.

本杰明·曼：是的，我认为这有一些细微差别，如果你看看朝鲜是如何赚取其经济收入的很大一部分的，那就是来自黑客加密货币交易所。

And if you look at, there's this Ben Buchanan book called The Hacker in The State that shows Russia did, it's almost like a live fire exercise where they just decided that they would shut down one of Ukraine's bigger power plants and from software destroy physical components in the power plant to make it harder to boot back up again.

如果你看一下，本·布坎南（Ben Buchanan）的这本名为《国家黑客》（The Hacker in the State）的书显示了俄罗斯的做法，这几乎就像一场实弹演习，他们刚刚决定关闭乌克兰最大的发电厂之一，并通过软件破坏发电厂的物理组件，使其更难以再次启动。

Benjamin Mann: And so I think people think of software as like, oh, it couldn't be that dangerous, but millions of people were without power for multiple days after that software attack.

本杰明·曼：所以我认为人们对软件的看法是，哦，它不可能那么危险，但在软件攻击后，数百万人连续几天断电。

I think there are real risks even when things are software only.

我认为即使只是软件，也存在真正的风险。

But I agree that when there's lots of robots running around, it gets, the stakes get even higher.

但我同意，当有很多机器人到处跑时，风险就会变得更高。

And I guess as a small push on this, Unitree is this Chinese company with these really amazing humanoid robots that cost $20,000 each, and they can do amazing things.

我想，作为对此的一个小小的推动，Unitree 是一家中国公司，拥有这些非常令人惊叹的人形机器人，每个售价 20,000 美元，而且它们可以做一些令人惊奇的事情。

They can do a standing back flip and manipulate objects, and the real thing that's missing there is the intelligence.

他们可以进行站立后空翻和操纵物体，但真正缺少的是智力。

And so the hardware is there and it's just going to get cheaper.

所以硬件已经有了，而且只会变得更便宜。

And I think in the next couple of years, it's like a pretty obvious question of whether the robot intelligence will make it viable soon.

我认为在接下来的几年里，一个非常明显的问题是机器人智能是否会很快使其变得可行。

Lenny Rachitsky: How much time do we have, Ben?

莱尼·拉奇茨基：本，我们还有多少时间？

What is your prediction of when this singularity hits until superintelligence starts to take off?

您对何时出现奇点直至超级智能开始起飞有何预测？

What's your prediction?

你的预测是什么？

Benjamin Mann: Yeah, I guess I mostly defer to the superforecasters here.

本杰明·曼：是的，我想我主要听从这里的超级预测家。

The AI 2027 report is probably the best one right now.

《AI 2027》报告可能是目前最好的一份。

Although ironically, their forecast is now 2028, and they didn't want to change the name of the thing-

虽然讽刺的是，他们的预测现在是2028年，而且他们不想改变这个东西的名字——

Lenny Rachitsky: The domain name, they already bought it.

Lenny Rachitsky：域名，他们已经买了。

Benjamin Mann: They already had the SEO.

本杰明·曼：他们已经进行了搜索引擎优化。

I think 50th percentile chance of hitting some kind of superintelligence in just a small handful of years is probably reasonable.

我认为在短短几年内实现某种超级智能的 50% 的可能性可能是合理的。

And it does sound crazy, but this is the exponential that we're on.

这听起来确实很疯狂，但这就是我们所处的指数。

It's not like a forecast that's pulled out of thin air.

这不像是凭空捏造的预测。

It's based on a lot of just hard details of the science of how intelligence seems to have been improving, the amount of low hanging fruit on model training, the scale ups of data centers and power around the world.

它基于许多科学的硬性细节，包括智力似乎是如何提高的、模型训练中容易实现的成果的数量、世界各地数据中心和电力的规模扩大。

I think it's probably a much more accurate forecast than people give it credit for.

我认为这可能是一个比人们想象的更准确的预测。

Benjamin Mann: I think if you had asked that same question 10 years ago, it would've been completely made up.

本杰明·曼：我认为如果你在 10 年前问同样的问题，那它就完全是编出来的。

Just the error bars were so high and we didn't have scaling laws back then and we didn't have techniques that seemed like they would get us there.

只是误差线太高了，而且我们当时没有缩放定律，也没有看起来能让我们实现这一目标的技术。

Times have changed, but I will repeat what I said earlier, which is even if we have superintelligence, I think it will take some time for its effects to be felt throughout society and the world.

时代变了，但我还是要重复我之前说过的话，即使我们拥有超级智能，我认为它的影响也需要一段时间才能让整个社会和全世界感受到。

And I think they'll be felt sooner and faster in some parts of the world than others.

我认为世界某些地区会比其他地区更快更快地感受到它们。

I think Arthur C.

我认为亚瑟·C.

Clark said, the future is already here, it's just not evenly distributed.

克拉克说，未来已经到来，只是分布不均。

Lenny Rachitsky: When we talk about this date of 2027, 2028, essentially it's when we start seeing superintelligence.

Lenny Rachitsky：当我们谈论 2027 年、2028 年这个日期时，本质上就是我们开始看到超级智能的时候。

Is there a way you think about what that...

有什么办法让你思考一下这是什么...

How do you define that?

你如何定义它？

Is it just all of a sudden AI's significantly smarter than the average human?

人工智能是否突然变得比普通人类聪明得多？

Is there another way you think about what that moment is?

您是否有另一种方式来思考那一刻是什么？

Benjamin Mann: Yeah, I think this comes back to the Economic Turing Test and seeing it pass for some sufficient number of jobs.

本杰明·曼：是的，我认为这又回到了经济图灵测试，并看到它通过了足够数量的工作。

Another way you could look at it though is if the world rate of GDP increase goes above 10% a year, then something really crazy must have happened.

另一种看待问题的方式是，如果世界 GDP 增长率每年超过 10%，那么一定发生了一些真正疯狂的事情。

I think we're at 3% now.

我想我们现在是 3%。

And so to see a 3X increase in that would be really game changing.

因此，看到这一数字增加 3 倍将真正改变游戏规则。

And if you imagine more than a 10% increase, it's very hard to even think about what that would mean from a individual story standpoint.

如果你想象超过 10% 的增长，就很难从单个故事的角度考虑这意味着什么。

If the amount of goods and services in the world is doubling every year, what does that even mean for me as a person living in California, let alone somebody living in some other part of the world that might be much worse off?

如果世界上的商品和服务数量每年翻一番，那么这对我作为一个生活在加利福尼亚州的人来说意味着什么，更不用说生活在世界其他地方的人了，情况可能会更糟？

Lenny Rachitsky: There's a lot of stuff here that's scary and I don't know how to think about it exactly.

Lenny Rachitsky：这里有很多令人恐惧的东西，我不知道如何准确地思考它们。

I'm hoping the answer to this is going to make me feel better.

我希望这个问题的答案能让我感觉好一点。

What are the odds that we align AI correctly and actually solve this problem, the stuff you're very much working on?

我们正确调整人工智能并真正解决这个问题（您正在努力解决的问题）的可能性有多大？

Benjamin Mann: It's a really hard question.

本杰明·曼：这是一个非常难的问题。

And there's really wide error bars.

而且误差线非常宽。

Anthropic has this blog post called Our Theory of Change or something like that, and it describes three different worlds, which is how hard is it to align AI.

Anthropic 有一篇名为“我们的变革理论”或类似内容的博客文章，它描述了三个不同的世界，这就是协调人工智能的难度。

There's a pessimistic world where it is basically impossible.

有一个悲观的世界，这基本上是不可能的。

There's an optimistic world where it's easy and it happens by default.

有一个乐观的世界，这很容易并且默认发生。

And then there's the world in between where our actions are extremely pivotal.

然后是介于两者之间的世界，我们的行动至关重要。

And I like this framing because it makes it a lot more clear what to actually do.

我喜欢这个框架，因为它让我们更清楚实际要做什么。

If we're in the pessimistic world, then our job is to prove that it is impossible to align safe AI and to get the world to slow down.

如果我们处于悲观的世界，那么我们的工作就是证明不可能调整安全的人工智能并让世界放慢速度。

Obviously that would be extremely hard.

显然，这将是极其困难的。

But I think we have some examples of coordination from nuclear non-proliferation and in general slowing down nuclear progress.

但我认为我们有一些在核不扩散和总体上减缓核进展方面进行协调的例子。

And I think that's the Doomer world basically.

我认为这基本上就是毁灭战士的世界。

And as a company, Anthropic doesn't have evidence that we're actually in that world yet, in fact, it seems like our alignment techniques are working.

作为一家公司，Anthropic 还没有证据表明我们确实处于那个世界，事实上，我们的对齐技术似乎正在发挥作用。

At least the prior on that is updating to be less likely.

至少之前更新的可能性较小。

Benjamin Mann: In the optimistic world, we're basically done, and our main job is to accelerate progress and to deliver the benefits to people.

本杰明·曼：在乐观的世界里，我们基本上已经完成了，我们的主要工作是加速进步并为人们带来好处。

But again, I think actually the evidence points against that world as well where we've seen evidence in the wild of deceptive alignment, for example, where the model will appear to be aligned but actually have some ulterior motive that it's trying to carry out in our laboratory settings.

但我再次认为，实际上证据也指向了那个世界，我们在欺骗性对齐的狂野中看到了证据，例如，模型看起来是对齐的，但实际上有一些别有用心的动机，它试图在我们的实验室环境中执行。

And so I think the world we're most likely in is this middle where alignment research actually does really matter.

所以我认为我们最有可能所处的世界是这个中间的世界，对齐研究实际上确实很重要。

And if we just do sort of the economically maximizing set of actions, then things will not go well.

如果我们只是采取经济最大化的一系列行动，那么事情就不会顺利。

Whether it's an X risk or just produces bad outcomes, I think is a bigger question.

我认为这是一个更大的问题，无论它是 X 风险还是只是产生不良结果。

Benjamin Mann: Taking it from that standpoint, I guess to state a thing about forecasting, people who haven't studied forecasting are bad at forecasting anything that's less than a 10% probability of happening.

本杰明·曼：从这个角度来看，我想说明一下有关预测的事情，没有研究过预测的人不擅长预测任何发生概率低于 10% 的事情。

And even those that have, it's quite a difficult skill, especially when there are few reference classes to lean on.

即使是那些拥有这些技能的人，这也是一项相当困难的技能，尤其是当几乎没有可供参考的参考课程时。

And in this case, I think there are very, very few reference classes for what an X risk kind of technology might look like.

在这种情况下，我认为关于 X 风险技术可能是什么样子的参考类非常非常少。

And so the way I think about it, I think my best granularity of forecasts for could we have an X risk or extremely bad outcome from AI is somewhere between 0 and 10%.

因此，我认为，对于人工智能是否会带来 X 风险或极其糟糕的结果，我的最佳预测粒度是在 0 到 10% 之间。

But from a marginal impact standpoint, as I said, since nobody is working on this, roughly speaking, I think it is extremely important to work on and that even if the world is likely to be a good one, that we should do our absolute best to make sure that that's true.

但从边际影响的角度来看，正如我所说，由于没有人在这方面努力，粗略地说，我认为努力是极其重要的，即使世界可能会变得美好，我们也应该尽最大努力确保这是真的。

Lenny Rachitsky: Wow.

莱尼·拉奇茨基：哇。

What fulfilling work.

多么有成就感的工作啊。

For folks that are inspired with this?

对于那些受到此启发的人？

I imagine you're hiring for folks to help you with this.

我想你正在招聘人来帮助你做这件事。

Maybe just share that in case folks are like, what can I do here?

也许只是分享一下，以防人们问我在这里能做什么？

Benjamin Mann: Yes.

本杰明·曼：是的。

I think 80,000 hours is the best guidance on this for a really detailed look into what do we need to make the field better?

我认为 80,000 小时是最好的指导，可以真正详细地了解我们需要做什么才能使该领域变得更好？

But a common misconception I see is that in order to have impact here, you have to be an AI researcher.

但我看到的一个常见的误解是，为了在这里产生影响，你必须成为一名人工智能研究员。

I personally actually don't do AI research anymore.

我个人实际上不再做人工智能研究了。

I work on product at Anthropic and product engineering, and we build things like Claude Code and Model Context Protocol, and a lot of the other stuff that people use every day.

我在 Anthropic 从事产品和产品工程工作，我们构建了 Claude Code 和 Model Context Protocol 等东西，以及人们每天使用的许多其他东西。

And that's really important because without an economic engine for our company to work on, and without being in people's hands all over the world, we won't have the mind policy influence and revenue to fund our future safety research and have the kind of influence that we need to have.

这非常重要，因为如果没有我们公司的经济引擎，也没有掌握在世界各地人们的手中，我们就没有思想政策影响力和收入来资助我们未来的安全研究，也没有我们需要的影响力。

If you work on product, if you work in finance, if you work in food, people here have to eat.

如果你从事产品工作，如果你从事金融工作，如果你从事食品工作，这里的人们就必须吃饭。

If you're a chef, we need all kinds of people.

如果你是厨师，我们需要各种各样的人。

Lenny Rachitsky: Awesome.

莱尼·拉奇茨基：太棒了。

Even if you're not working directly on the AI safety team, you're having an impact on moving things in the right direction.

即使您不直接在人工智能安全团队工作，您也会对推动事情朝着正确的方向发展产生影响。

By the way, X risk is short for existential risk.

顺便说一下，X 风险是存在风险的缩写。

In case folks haven't heard that term.

如果人们还没有听说过这个词。

I have a few random questions along these lines and then I want to zoom out again.

我有一些随机的问题，然后我想再次缩小。

You mentioned this idea of AI being aligned using its model, like reinforcing itself.

您提到了人工智能使用其模型进行调整的想法，例如增强自身。

You have this term RLAIF.

您有这个术语 RLAIF。

Is that what that describes?

是这样描述的吗？

Benjamin Mann: Yeah.

本杰明·曼：是的。

RLAIF is reinforcement learning from AI feedback.

RLAIF 是根据人工智能反馈进行强化学习。

Lenny Rachitsky: People have heard of RLHF, reinforcement learning with human feedback.

Lenny Rachitsky：人们听说过 RLHF，即带有人类反馈的强化学习。

I don't think a lot of people have heard this.

我想很多人都没有听说过这个。

Talk about just the significance of this shift you guys have made in training your models.

谈谈你们在训练模型时所做的转变的重要性。

Benjamin Mann: Yeah, so RLAIF, constitutional AI is an example of this where there are no humans in the loop, and yet the AI is sort of self-improving in ways that we want it to.

本杰明·曼：是的，RLAIF 宪法人工智能就是一个例子，其中循环中没有人类，但人工智能却以我们希望的方式进行自我改进。

And another example of RLAIF is if you have models writing code and other models commenting on various aspects of what that code looks like of is it maintainable, is it correct, does it pass the linter?

RLAIF 的另一个例子是，如果您有编写代码的模型，而其他模型则对代码的各个方面进行评论，例如代码是否可维护、是否正确、是否通过了 linter？

Things like that.

诸如此类的事情。

That also could be included in RLAIF.

这也可以包含在 RLAIF 中。

And the idea here is that if models can self-improve, then it's a lot more scalable than finding a lot of humans.

这里的想法是，如果模型可以自我改进，那么它比寻找大量人类更具可扩展性。

Ultimately, people think about this as probably going to hit a wall because if the model isn't good enough to see its own mistakes, then how could it improve?

最终，人们认为这可能会碰壁，因为如果模型不够好，无法看到自己的错误，那么它如何改进呢？

And also, if you read the AI 2027 story, there's a lot of risk of if the model is in a box trying to improve itself, then it could go completely off the rails and have these secret goals like resource accumulation and power seeking and resistance to shut down that you really don't want in a very powerful model.

而且，如果你读过 AI 2027 的故事，就会发现，如果模型被装在一个盒子里试图自我改进，那么它可能会完全偏离轨道，并实现诸如资源积累、权力寻求和抵制关闭等秘密目标，而这些目标是你在一个非常强大的模型中真正不想要的。

And we've actually seen that in some of our experiments in laboratory settings.

事实上，我们在实验室环境中的一些实验中已经看到了这一点。

Benjamin Mann: How do you do recursive self-improvement and make sure it's aligned at the same time?

本杰明·曼：你如何进行递归自我改进并确保它同时保持一致？

I think that's the name of the game.

我想这就是游戏的名字。

To me, it just nets out to how do humans do that and how do human organizations do that?

对我来说，它只是解决人类如何做到这一点以及人类组织如何做到这一点？

Corporations are probably the most scaled human agents today.

公司可能是当今规模最大的人类代理人。

They have certain goals that they're trying to reach, and they have certain guiding principles, they have some oversight in terms of shareholders and stakeholders and board members.

他们有一定的目标，他们试图达到，他们有一定的指导原则，他们对股东、利益相关者和董事会成员有一定的监督。

How do you make corporations aligned and able to sort of recursively self-improve?

如何使公司保持一致并能够递归地自我改进？

Benjamin Mann: And another model to look at is science, where the purpose of science is to do things that have never been done before and push the frontier.

本杰明·曼：另一个值得关注的模型是科学，科学的目的是做以前从未做过的事情并推动前沿。

And to me, it all comes down to empiricism.

对我来说，这一切都归结为经验主义。

When people don't know what the truth is, they come up with theories and then they design experiments to try them out.

当人们不知道真相是什么时，他们会提出理论，然后设计实验来验证它们。

And similarly, if we can give models those same tools, then we could expect them to sort of improve recursively in an environment and potentially become much better than humans could be just by banging their head against reality or I guess metaphorical head.

同样，如果我们可以为模型提供相同的工具，那么我们可以期望它们在环境中递归地改进，并且有可能变得比人类通过用头撞现实或我猜是隐喻的头更好。

Benjamin Mann: I guess I don't expect there to be a wall in terms of model's ability to improve themselves if we can give them access to the ability to be empirical.

本杰明·曼（Benjamin Mann）：我想如果我们能让模型获得实证能力，那么我并不认为模型自我改进的能力会存在障碍。

And I guess Anthropic, deeply in its DNA is an empirical company.

我想 Anthropic 从本质上来说是一家经验主义公司。

We have a lot of physicists like Jared, who's our chief research officer who I've worked with a lot, was a professor of Black Hole Physics at Johns Hopkins, and I guess he technically still is, but on leave.

我们有很多像贾里德这样的物理学家，他是我们的首席研究官，我和他一起工作过很多次，他是约翰·霍普金斯大学的黑洞物理学教授，我想从技术上来说他仍然是，但正在休假。

Yeah, it's in our DNA and yeah, I guess that's the RLAIF.

是的，这是我们的 DNA，是的，我想这就是 RLAIF。

Lenny Rachitsky: Let me just follow this thread on, in terms of bottleneck, this is kind of a tangent, but just what is the biggest bottleneck today on model intelligence improvement?

Lenny Rachitsky：让我继续这个思路，就瓶颈而言，这是一种切线，但当今模型智能改进的最大瓶颈是什么？

Benjamin Mann: The stupid answer is data centers and power chips.

本杰明·曼：愚蠢的答案是数据中心和电源芯片。

I think if we had 10 times as many chips and had the data centers to power them, then maybe we wouldn't go 10 times faster, but it would be a real significant speed boost.

我认为，如果我们拥有 10 倍数量的芯片并拥有为其供电的数据中心，那么也许我们的速度不会快 10 倍，但这将是真正显着的速度提升。

Lenny Rachitsky: It's actually very much scaling loss, just more compute.

Lenny Rachitsky：这实际上是很大的缩放损失，只是更多的计算。

Benjamin Mann: Yeah, I think that's a big one.

本杰明·曼：是的，我认为这是一件大事。

And then the people really matter.

人真的很重要。

We have great researchers and many of them have made really significant contributions to the science of how the models improve.

我们拥有优秀的研究人员，其中许多人为模型如何改进的科学做出了非常重大的贡献。

And so it's like compute, algorithms, and data.

这就像计算、算法和数据。

Those are the three ingredients in the scaling laws.

这些是缩放定律中的三个要素。

And just to make that concrete, before we had transformers, we had LSTMs and we've done scaling laws on what the exponent is on those two things.

为了具体说明这一点，在我们有了 Transformer 之前，我们有了 LSTM，并且我们已经对这两件事的指数制定了缩放法则。

And we found that for transformers, the exponent is higher.

我们发现对于变压器来说，指数更高。

And making changes like that where as you increase scale, you also increase your ability to squeeze out intelligence.

做出这样的改变，随着规模的扩大，你挤出智慧的能力也随之增强。

Those kinds of things are super impactful.

这类事情非常有影响力。

Benjamin Mann: And so having more researchers who can do better science and find out how do we squeeze out more gains is another one.

本杰明·曼：因此，拥有更多的研究人员可以做更好的科学研究并找出我们如何获得更多收益是另一回事。

And then with the rise of reinforcement learning, the efficiency with which these things run on chips also matters a lot.

随着强化学习的兴起，这些东西在芯片上运行的效率也很重要。

We've seen in the industry a 10X decrease in cost for a given amount of intelligence through a combination of algorithmic data and efficiency improvements.

我们在行业中看到，通过算法数据和效率改进的结合，一定数量的智能的成本降低了 10 倍。

And if that continues, in three years we'll have 1,000 deck smarter models for the same price.

如果这种情况持续下去，三年内我们将以相同的价格推出 1,000 种更智能的甲板模型。

Kind of hard to imagine,

有点难以想象，

Lenny Rachitsky: I forget where I heard this, but it's amazing that so many innovations came together at the same time to allow for this sort of thing and continue to progress where one thing isn't just slowing everything down like we're out of some rare earth mineral or we just can't optimize reinforcement learning more.

Lenny Rachitsky：我忘记了我在哪里听到的，但令人惊讶的是，如此多的创新同时聚集在一起，允许这种事情发生，并继续取得进展，其中一件事不仅仅是减慢一切，就像我们耗尽了某种稀土矿物，或者我们只是无法进一步优化强化学习。

It's amazing that we continue to find improvements and there isn't one thing that's just slowing everything down.

令人惊奇的是，我们不断寻求改进，而且没有任何一件事会拖慢一切。

Benjamin Mann: Yeah, I think it really is just a combination of everything probably will hit a wall at some point.

本杰明·曼：是的，我认为这确实是所有事情的结合，可能会在某个时候碰壁。

I guess in semiconductors.

我猜是在半导体领域。

My brother works in the semiconductor industry and he was telling me that you can't actually shrink the size of the transistors anymore because the way semiconductors work is you dope silicon with other elements and the doping process would result in either zero or one atom of the doped elements inside a single fin because they're so, so, so tiny.

我的兄弟在半导体行业工作，他告诉我，你实际上不能再缩小晶体管的尺寸，因为半导体的工作方式是用其他元素掺杂硅，而掺杂过程会导致单个鳍内掺杂元素的零个或一个原子，因为它们是如此、如此、如此微小。

Benjamin Mann: And that's just wild to think of, and yet Moore's law somehow continues in some form.

本杰明·曼：这真是令人难以置信，但摩尔定律却以某种形式继续存在。

And so yes, there are these theoretical physics constraints that people are starting to run into and yet they're finding ways around it.

是的，人们开始遇到这些理论物理限制，但他们正在寻找解决方法。

Lenny Rachitsky: We've got to start using parallel universes for some of this stuff.

Lenny Rachitsky：我们必须开始使用平行宇宙来处理其中的一些东西。

Lenny Rachitsky: Okay, I want to zoom out and talk about just Ben, Ben as a human for a moment before we get to a very exciting lightning round.

Lenny Rachitsky：好的，在我们进入非常激动人心的闪电回合之前，我想先把镜头拉远，谈谈本，本作为一个人。

I imagine just kind of the burden of feeling responsible for safe superintelligence is a heavy one.

我想，对安全超级智能负责的感觉是一种沉重的负担。

It feels like you're in a place where you can make a significant impact on the future of safety and AI.

感觉就像你正处于一个可以对安全和人工智能的未来产生重大影响的地方。

That's a lot of weight to carry.

承载的重量很大。

How does that just impact you personally, impact your life, how you see the world?

这对你个人、你的生活以及你如何看待世界有何影响？

Benjamin Mann: There's this book that I read in 2019 that really informs how I think about sort of working with these very weighty topics called Replacing Guilt by Nate Soares.

本杰明·曼（Benjamin Mann）：我在 2019 年读过这本书，它确实让我了解了如何处理这些非常重要的主题，即内特·苏亚雷斯（Nate Soares）的《取代内疚》（Replacing Guilt）。

And he describes a lot of different techniques for kind of working through this kind of thing.

他描述了许多不同的技术来解决此类问题。

And he's actually the executive director at MIRI, the Machine Intelligence Research Institute, which is an AI safety tank that I worked at for a couple of months actually.

他实际上是 MIRI（机器智能研究所）的执行董事，实际上我在该研究所工作了几个月。

And one of the things he talks about is this thing called resting in motion where some people think that the default state is rest, but actually that was never in the state of evolutionary adaptation.

他谈到的一件事是所谓的“运动中的休息”，有些人认为默认状态是休息，但实际上这从来都不是进化适应的状态。

I really doubt that that was true.

我真的怀疑那是真的。

Where in nature, in the wilderness being hunter-gatherers and it's really unlikely that we evolved to just be at leisure, probably always have something to worry about of defending the tribe and finding enough food to survive and taking care of the children, dealing-

在大自然中，在荒野中，我们是狩猎采集者，我们不太可能进化到只是休闲，可能总是需要担心保卫部落、寻找足够的食物来生存、照顾孩子、处理——

Benjamin Mann: And so I think about that as the busy state is the normal state and to try to work at a sustainable pace that it's a marathon, not a sprint, that's one thing that helps.

本杰明·曼：所以我认为忙碌的状态是正常的状态，并且尝试以可持续的速度工作，这是一场马拉松，而不是短跑，这是有帮助的一件事。

And then just being around like-minded people that also care.

然后就和志同道合、也关心我们的人在一起。

It's not a thing that any of us can do alone.

这不是我们任何人能够单独完成的事情。

And Anthropic has incredible talent density.

Anthropic 拥有令人难以置信的人才密度。

One of the things I love the most about our culture here is that it's very egoless.

我最喜欢这里文化的一点是它非常无私。

People just want the right thing to happen and I think that's another big reason that the mega offers from other companies tend to bounce off because people just love being here and they care.

人们只是希望发生正确的事情，我认为这是其他公司的巨额报价往往会反弹的另一个重要原因，因为人们只是喜欢在这里并且他们关心。

Lenny Rachitsky: That's amazing.

莱尼·拉奇茨基：太棒了。

I don't know how you do it.

我不知道你是怎么做到的。

I'd be extremely stressed.

我会感到非常压力。

I'm going to try this resting in motion strategy.

我将尝试这种静止的运动策略。

Okay, so you've been at Anthropic for a long time.

好吧，你已经在 Anthropic 工作很长时间了。

From the very beginning I was reading there were 7 employees back in 2020.

从一开始我就读到 2020 年有 7 名员工。

Today there's over 1,000, I don't know what the latest number is, but I know it's over 1,000.

今天有1000多了，我不知道最新的数字是多少，但我知道是1000多了。

I've heard also that you've done basically every job at Anthropic, you made big contributions to a lot of the core products, the brand, the team hiring.

我还听说你基本上完成了 Anthropic 的所有工作，你对很多核心产品、品牌、团队招聘做出了巨大贡献。

Let me just ask I guess what's most changed over that period?

让我问一下，我猜那段时间变化最大的是什么？

What is most different from the beginning days and which of those jobs that you've had over the years have you most loved?

与刚开始的时候相比，什么是最不同的？这些年来，您最喜欢哪一份工作？

Benjamin Mann: I probably had 15 different roles, honestly.

本杰明·曼：老实说，我可能扮演过 15 个不同的角色。

I was head of security for a bit.

我曾担任过一段时间的安全主管。

I managed the Ops team when our president was on mat leave, I was crawling around under tables, plugging in HDMI cords and doing pen testing on our building.

当我们的总裁休假时，我负责管理运营团队，我在桌子底下爬来爬去，插入 HDMI 线，并对我们的大楼进行渗透测试。

And I started our product team from scratch and convinced the whole company that we needed to have a product instead of just being a research company.

我从头开始组建了我们的产品团队，并说服整个公司，我们需要有一个产品，而不仅仅是一家研究公司。

Yeah, it's been a lot.

是的，已经很多了。

All of it very fun.

所有这一切都非常有趣。

I think my favorite role in that time has been when I started the labs team about a year ago, whose fundamental goal was to do transfer from research to end user products and experiences.

我认为那段时间我最喜欢的角色是大约一年前我组建实验室团队时，其基本目标是从研究转移到最终用户产品和体验。

Because fundamentally I think the way that Anthropic can differentiate itself and really win is to be on the cutting edge.

因为从根本上来说，我认为 Anthropic 能够脱颖而出并真正获胜的方式就是走在最前沿。

We have access to the latest, greatest stuff that's happening and I think honestly through our safety research we have a big opportunity to do things that no other company can safely do.

我们可以了解正在发生的最新、最伟大的事情，老实说，通过我们的安全研究，我们有很大的机会去做其他公司无法安全做的事情。

Benjamin Mann: For example, with computer use, I think that's going to be our huge opportunity basically to make it possible for an agent to use all your credentials on your computer, there has to be a huge amount of trust and to me we need to basically solve safety to make that happen.

本杰明·曼：例如，在计算机使用方面，我认为这将是我们的巨大机会，基本上使代理能够使用您计算机上的所有凭据，必须有巨大的信任，对我来说，我们需要基本上解决安全问题才能实现这一点。

Safety and alignment.

安全和对齐。

I'm pretty bullish on that kind of thing and I think we're going to see really cool stuff coming out soonish.

我非常看好这类事情，我认为我们很快就会看到非常酷的东西问世。

Yeah, just leading that team has been so fun.

是的，领导这个团队真是太有趣了。

MCP came out of that team and Claude Code came out of that team.

MCP 出自该团队，Claude Code 出自该团队。

And the people who I hired are like combo, have been a founder and also have been at big companies and seeing how things work at scale.

我雇用的人就像组合一样，曾经是创始人，也曾在大公司工作过，了解事情如何大规模运作。

It's just been an incredible team to work with and figure out the future with.

这是一个令人难以置信的团队，可以与之合作并共同规划未来。

Lenny Rachitsky: I want to hear more about this.

Lenny Rachitsky：我想听听更多相关信息。

Team actually the person that connected us, the reason we're doing this is a mutual friend colleague Raph Lee who I used to work with at Airbnb now works on this team, leads a lot of this work and so he wanted me to make sure I asked about this team because...

团队实际上是联系我们的人，我们这样做的原因是我共同的朋友同事 Raph Lee，我曾经在 Airbnb 共事过，现在在这个团队工作，领导了很多这项工作，所以他希望我确保我询问过这个团队，因为......

I didn't realize all these things came out that team.

我没有意识到所有这些事情都是那个团队出来的。

Holy moly.

圣钼。

What else should people know about this team?

人们还应该了解这个团队什么？

It used to be called Labs, I think it's called Frontiers now.

它曾经被称为Labs，我想它现在被称为Frontiers。

Lenny Rachitsky: Cool.

莱尼·拉奇茨基：酷。

The idea here is this team works with the latest technologies that you guys have built and explores what is possible.

这里的想法是这个团队使用你们建立的最新技术并探索可能性。

Is that the general idea?

这是一般的想法吗？

Benjamin Mann: Yeah, and I guess I was part of Google's Area 120 and I've read about Bell Labs and how to make these innovation teams work.

本杰明·曼：是的，我想我是 Google 120 区的一员，我读过有关贝尔实验室以及如何让这些创新团队开展工作的信息。

It's really hard to do right and I wouldn't say that we've done everything right, but I think we've done some serious innovation on the state-of-the-art from company design and Raph has been right at the center of that.

要做对确实很难，我不会说我们已经把所有事情都做对了，但我认为我们已经在公司设计方面对最先进的技术进行了一些认真的创新，而 Raph 一直处于其中的中心。

When I was first fitting up the team, the first thing I did was hire a great manager and that was Raph.

当我第一次组建团队时，我做的第一件事就是聘请一位出色的经理，那就是拉夫。

And so he's definitely been crucial in building the team and helping it operate well.

因此，他在组建团队并帮助其良好运作方面无疑发挥了至关重要的作用。

And we defined some operating models like the journey of an idea from prototype to product and how should graduation of products and projects work, how do teams do sprint models that are effective and make sure that they're working on the right ambition level of thing.

我们定义了一些运营模型，例如创意从原型到产品的旅程，以及产品和项目的毕业应该如何工作，团队如何进行有效的冲刺模型，并确保他们正在朝着正确的目标水平努力。

That's been really exciting.

这真的很令人兴奋。

Benjamin Mann: I guess concretely we think about skating to where the puck is going and what that looks like is really understand the exponential.

本杰明·曼：我想具体来说，我们会考虑滑到冰球要去的地方，这看起来就像真正理解指数一样。

There's this great study that METR has done that Beth Barnes is the CEO of that organization and shows how long a time horizon of software engineering task can be done and just really internalizing that of, okay, don't build for today, build for six months from now, build for a year from now.

METR 做了一项很棒的研究，贝丝·巴恩斯（Beth Barnes）是该组织的首席执行官，它展示了软件工程任务可以完成多长时间，并且真正内化了这一点，好吧，不要为今天构建，从现在开始六个月构建，从现在开始构建一年。

And the things that aren't quite working that are working 20% of the time, will start working 100% of the time.

那些在 20% 的时间内工作得不太好的东西，将在 100% 的时间内开始工作。

And I think that's really what made Claude Code a success that we thought people are not going to be locked to their IDEs forever.

我认为这正是 Claude Code 成功的原因，我们认为人们不会永远局限于他们的 IDE。

People are not going to be auto completing.

人们不会自动完成。

People will be doing everything that a software engineer needs to do and a terminal is a great place to do that because a terminal can live in lots of places.

人们将做软件工程师需要做的一切，而终端是一个很好的地方，因为终端可以驻留在很多地方。

A terminal can live on your local machine, it can live in GitHub actions, it can live on a remote machine in your cluster.

终端可以驻留在本地计算机上，可以驻留在 GitHub 操作中，也可以驻留在集群中的远程计算机上。

Benjamin Mann: That's sort of the leverage point for us and that was a lot of the inspiration.

本杰明·曼：这对我们来说是一种杠杆点，也是很多灵感的来源。

I think that's what the labs team tries to think about.

我认为这就是实验室团队试图思考的问题。

Are we AGI-pilled enough?

我们的 AGI 足够了吗？

Lenny Rachitsky: What a fun place to be.

Lenny Rachitsky：多么有趣的地方啊。

By the way, fun fact, Raph was my first manager at Airbnb when I joined.

顺便说一句，有趣的是，当我加入 Airbnb 时，Raph 是我在 Airbnb 的第一位经理。

I was an engineer and he was my first manager.

我是一名工程师，他是我的第一位经理。

It all worked out.

一切顺利。

Lenny Rachitsky: Yeah.

莱尼·拉奇茨基：是的。

Okay.

好的。

Final question before the very exciting lighting round.

在非常激动人心的灯光回合之前的最后一个问题。

I've never asked this question before.

我以前从未问过这个问题。

I'm curious what your answer would be if you could ask a future AGI one single question and be guaranteed to get the right answer, what would you ask?

我很好奇你的答案是什么，如果你可以向未来的 AGI 提出一个问题并保证得到正确的答案，你会问什么？

Benjamin Mann: I have two dumb answers.

本杰明·曼：我有两个愚蠢的答案。

First for fun.

首先是为了好玩。

Benjamin Mann: The first is there's this Asimov short story I love called the last question where the protagonist is throughout the eras of history is trying to ask this super intelligence how do we prevent the heat death of the universe?

本杰明·曼：首先是我喜欢的阿西莫夫短篇小说，它被称为最后一个问题，主角在整个历史时代试图问这个超级智慧，我们如何防止宇宙的热寂？

And I won't spoil the ending, but it's a fun question.

我不会破坏结局，但这是一个有趣的问题。

Lenny Rachitsky: You would ask it that question because the one in the story was unsatisfying?

Lenny Rachitsky：你会问这个问题是因为故事中的内容不令人满意？

Benjamin Mann: Okay, I'll give it away.

本杰明·曼：好的，我会把它放弃的。

It keeps saying, "Need more information, need more compute." And then finally, as it's approaching the heat death of the universe, it says, "Let there be light," and then it starts the universe over again.

它一直在说：“需要更多信息，需要更多计算。”最后，当它接近宇宙的热寂时，它说：“要有光”，然后它再次开始宇宙。

Lenny Rachitsky: Oh wow.

莱尼·拉奇茨基：哦，哇。

That's beautiful.

那好美丽。

That's beautiful.

那好美丽。

Benjamin Mann: That's the first cheat answer.

本杰明·曼：这是第一个作弊答案。

The second cheat answer is what question can I ask you to get end more questions answered.

第二个作弊答案是我可以问你什么问题才能得到更多问题的答案。

Benjamin Mann: And then the third answer, which is my real question is how do we ensure the continued flourishing of humanity into the indefinite future?

本杰明·曼：然后第三个答案，也是我真正的问题是，我们如何确保人类在无限期的未来持续繁荣？

That's the question I'd love to know and if I can be guaranteed a correct answer then seems very valuable to ask.

这是我很想知道的问题，如果我能保证得到正确的答案，那么问这个问题似乎很有价值。

Lenny Rachitsky: I wonder what would happen if you ask a lot that today and then how that answer changes over the next couple years.

Lenny Rachitsky：我想知道如果你今天问了很多这个问题，然后这个答案在接下来的几年里会发生什么变化。

Benjamin Mann: Yeah, maybe I'll try that.

本杰明·曼：是的，也许我会尝试一下。

I'll put it into the deep research thing that we have and see what it comes out with.

我会将其投入到我们现有的深入研究中，看看会产生什么结果。

Lenny Rachitsky: Okay.

莱尼·拉奇茨基：好的。

I'm excited to see what you come up with.

我很高兴看到你的想法。

Ben, is there anything else you wanted to mention or leave listeners with maybe as a final nugget before we get to our very exciting lightning round?

Ben，在我们进入非常激动人心的闪电回合之前，您还有什么想提的，或者作为最后的要点留给听众吗？

Benjamin Mann: Yeah, I guess my push would be these are wild times.

本杰明·曼：是的，我想我的推动力是现在是疯狂的时代。

If they don't seem wild to you, then you must be living under a rock but also get used to it because this is as normal as it's going to be.

如果它们对你来说并不狂野，那么你一定是生活在岩石下，但也要习惯它，因为这很正常。

It's going to be much weirder very soon.

很快就会变得更加奇怪。

And if you can sort of mentally prepare yourself for that, I think you'll be better off.

如果你能为此做好心理准备，我想你会过得更好。

Lenny Rachitsky: I need to make that the title of this episode.

Lenny Rachitsky：我需要将其作为本集的标题。

It's going to get much weirder very soon.

很快事情就会变得更加奇怪。

I 100% believe that.

我百分百相信这一点。

Oh my God.

我的天啊。

I don't know what's in store.

我不知道店里有什么。

I love how you're at the center of it all.

我喜欢你处于这一切的中心。

With that, we reached our very exciting lightning round.

就这样，我们进入了非常激动人心的闪电回合。

I've got five questions for you.

我有五个问题要问你。

Are you ready?

你准备好了吗？

Lenny Rachitsky: What are two or three books that you find yourself recommending most to other people?

Lenny Rachitsky：您最常向其他人推荐的两三本书是什么？

Benjamin Mann: The first one I mentioned before, Replacing Guilt by Nate Soares.

本杰明·曼：我之前提到的第一个，内特·苏亚雷斯的《取代内疚》。

Love that one.

喜欢那个。

The second one is Good Strategy Bad Strategy by Richard Rumelt.

第二本是理查德·鲁梅尔特（Richard Rumelt）的《好策略、坏策略》。

Just thinking about in a very clear way, how do you build product?

只是以一种非常清晰的方式思考，你如何构建产品？

It's one of the best strategy books I've read and strategy is a hard word to even think about in many ways.

这是我读过的最好的战略书籍之一，而战略这个词在很多方面都是一个很难思考的词。

And then the last one is The Alignment Problem by Brian Christian.

最后一个是 Brian Christian 的《对齐问题》。

Just really thoughtfully goes through what is this problem that we care about that we're trying to solve here?

只是真正深思熟虑地思考我们关心的、我们在这里试图解决的问题是什么？

What are the stakes in a version that's more updated and easier to read and digest than superintelligence?

比超级智能更更新、更容易阅读和消化的版本有什么风险？

Lenny Rachitsky: I've got Good Strategy, Bad Strategy right behind me.

Lenny Rachitsky：我身后有好策略和坏策略。

I think I'm going to point to it.

我想我会指出这一点。

There it is.

就在那里。

Lenny Rachitsky: And I've had Richard Rumelt on the podcast in case anyone wants to hear from him directly.

Lenny Rachitsky：我在播客上邀请了 Richard Rumelt，以防有人想直接听到他的消息。

Next question, do you have a favorite recent movie or TV show you've really enjoyed?

下一个问题，您最近有最喜欢的电影或电视节目吗？

Benjamin Mann: Pantheon was really good based on Ken Liu or Ted Chiang's story.

本杰明·曼：根据刘宇昆或特德·姜的故事，万神殿真的很棒。

Ken Liu I think.

我想刘肯。

Super good talks about what does it mean if we have uploaded intelligences and what are their moral and ethical exigencies.

超级好谈论如果我们上传情报意味着什么以及他们的道德和伦理紧迫性是什么。

Ted Lasso, which is supposedly about soccer, but actually it's about human relationships and how people get along and just super heartwarming and funny.

泰德·拉索（Ted Lasso），据说是关于足球的，但实际上它是关于人际关系以及人们如何相处的，非常温暖和有趣。

And then this isn't really a TV show, but Kurzgesagt is my favorite YouTube channel and goes through random science and social problems and is just super well done and super well-made.

这并不是真正的电视节目，但 Kurzgesagt 是我最喜欢的 YouTube 频道，它讲述了随机的科学和社会问题，而且做得非常好，制作也非常精良。

Love watching that.

喜欢看那个。

Lenny Rachitsky: Wow.

莱尼·拉奇茨基：哇。

Haven't heard of that as you were talking, I feel like Ted Lasso, I feel like that's what you need to put into constitutional AI, act like Ted Lasso.

没有听说过你所说的，我感觉就像特德·拉索，我觉得这就是你需要将其放入宪法人工智能中的东西，像特德·拉索一样行事。

Lenny Rachitsky: ...

莱尼·拉奇茨基：……

Hardworking.

勤奋。

Oh my God.

我的天啊。

There we go.

我们开始吧。

I think we've solved alignment problems right here.

我认为我们已经解决了这里的对齐问题。

Get those writers on this, ASAP.

尽快让那些作家关注这个问题。

Okay, two more questions.

好吧，还有两个问题。

Do you have a favorite life motto that you often come back to in work or in life?

您在工作或生活中是否有最喜欢的人生格言？

Benjamin Mann: Well, a really dumb one is, have you tried asking Claude?

本杰明·曼：嗯，一个非常愚蠢的问题是，你试过问克劳德吗？

And this is getting more and more common where recently I asked a coworker like, "Hey, who's working on X?" And they were like, "Let me Claude that for you." And then they sent me the link to the thing afterwards and I was like, "Oh yeah, thanks.

这种情况变得越来越普遍，最近我问一位同事，“嘿，谁在开发 X？”他们就像是，“让我为你克劳德。”然后他们给我发了那个东西的链接，我当时想，“哦，是的，谢谢。

That's great." But maybe more of a philosophical one I would say, everything is hard.

那太好了。”但也许我会说更多的是一种哲学，一切都是艰难的。

Just to remind ourselves that things that feel like they're supposed to be easy, it's okay to not be easy and sometimes you just have to push through anyway.

只是提醒自己，那些感觉应该很容易的事情，并不容易也没关系，有时你无论如何都必须坚持下去。

Lenny Rachitsky: And rest in motion while you're doing that.

Lenny Rachitsky：当你这样做的时候，要在运动中休息。

Lenny Rachitsky: Final question.

莱尼·拉奇茨基：最后一个问题。

I don't know if you want people to know this, but I was browsing through your Medium posts and you have a post called Five Tips to Poop like a Champion.

我不知道你是否想让人们知道这一点，但我浏览了你的 Medium 帖子，你有一篇名为“像冠军一样大便的五个技巧”的帖子。

I'd love it.

我会喜欢的。

Can you share one tip to poop like a champion if you remember your tips?

如果您还记得您的秘诀，您能分享一个像冠军一样大便的秘诀吗？

Benjamin Mann: I of course do.

本杰明·曼：我当然知道。

It's actually my most popular Medium posts.

这实际上是我最受欢迎的 Medium 帖子。

Lenny Rachitsky: Okay, great.

莱尼·拉奇茨基：好的，太好了。

I can see that.

我看得出来。

It's a great title.

这是一个很棒的标题。

Benjamin Mann: I think maybe my biggest tip would be use a bidet.

本杰明·曼：我想我最大的建议可能是使用坐浴盆。

It's amazing.

太棒了。

It's life-changing.

它改变了生活。

It's so good.

太好了。

Some people are kind of freaked out by it.

有些人有点被它吓坏了。

It's the standard in countries like Japan and I think it's just more civilized.

这是日本等国家的标准，我认为它更加文明。

And in 10 or 20 years people would be like, how could you not use that?

10 年或 20 年后，人们会想，你怎么能不使用它呢？

Lenny Rachitsky: And a bidet could be like a Japanese toilet.

Lenny Rachitsky：坐浴盆可能就像日本马桶。

That's along the same lines.

这也是同样的道理。

Lenny Rachitsky: Right.

莱尼·拉奇茨基：对。

Okay.

好的。

I love where we went with this.

我喜欢我们这样做的地方。

Ben, this was incredible.

本，这太不可思议了。

Thank you so much for doing this.

非常感谢您这样做。

Thank you so much for sharing so much real talk.

非常感谢您分享这么多真实的谈话。

Two final questions.

最后两个问题。

Where can folks find you online if they want to reach out, maybe go work at Anthropic and how can listeners be useful to you?

如果人们想联系您，可以在哪里找到您，也许可以去 Anthropic 工作，听众如何对您有用？

Benjamin Mann: You can find me online at benjmann.net and on our website, we have a great careers page that we're working on making a little bit easier to access and figure out, but definitely point Claude at it and it can help you figure out what could be interesting for you.

本杰明·曼：你可以在 benjmann.net 和我们的网站上找到我，我们有一个很棒的职业页面，我们正在努力使该页面更容易访问和了解，但一定要向克劳德指出它，它可以帮助你找出你感兴趣的东西。

And how can listeners be useful to me?

听众如何对我有用？

I think safety pill yourself, that's the number one thing and spread it to your network.

我认为自己服用安全药丸，这是第一要务，并将其传播到您的网络中。

I think.

我认为。

Like I said, there are very few people working on this and it's so important.

就像我说的，很少有人从事这方面的工作，但这非常重要。

Yeah, think hard about it and try to look at it.

是的，认真思考一下并尝试看看。

Lenny Rachitsky: Thanks for spreading the gospel, Ben, thank you so much for being here.

Lenny Rachitsky：谢谢你传播福音，Ben，非常感谢你来到这里。

Lenny Rachitsky: Bye everyone.

莱尼·拉奇茨基：大家再见。

Thank you so much for listening.

非常感谢您的聆听。

If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.

如果您发现这很有价值，您可以在 Apple Podcasts、Spotify 或您最喜欢的播客应用程序上订阅该节目。

Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast.

另外，请考虑给我们评分或留下评论，因为这确实有助于其他听众找到播客。

You can find all past episodes or learn more about the show at lennyspodcast.com.

您可以在 lennyspodcast.com 上找到所有过去的剧集或了解有关该节目的更多信息。

See you in the next episode.

下一集见。