The $1B Al company training ChatGPT, Claude & Gemini on the path to responsible AGI | Edwin Chen

价值 10 亿美元的 Al 公司培训 ChatGPT、Claude 和 Gemini，走上负责任的 AGI 之路 |埃德温·陈

2025-12-07 · Lenny's Podcast · 原文

Lenny Rachitsky: You guys hit a billion in revenue in less than four years with around 60 to 70 people.

Lenny Rachitsky：你们在不到四年的时间里凭借大约 60 到 70 名员工实现了 10 亿美元的收入。

You're completely bootstrapped, haven't raised any VC money.

你完全是自力更生，没有筹集到任何风险投资资金。

I don't believe anyone has ever done this before.

我相信以前没有人这样做过。

Edwin Chen: We basically never wanted to play the Silicon Valley game.

Edwin Chen：我们基本上从来不想玩硅谷游戏。

I always thought it was ridiculous.

我一直觉得这很可笑。

I used to work at a bunch of the big tech companies and I always felt that we could fire 90% of the people and we would move faster because the best people wouldn't have all these distractions.

我曾经在多家大型科技公司工作过，我总觉得我们可以解雇 90% 的员工，而且我们会采取更快的行动，因为最优秀的人不会受到这些干扰。

So when we start Surge, we wanted to build it completely differently with a super small, super elite team.

因此，当我们启动 Surge 时，我们希望通过一个超小、超精英的团队以完全不同的方式构建它。

Lenny Rachitsky: You guys are by far the most successful data company out there.

Lenny Rachitsky：你们是迄今为止最成功的数据公司。

Edwin Chen: We essentially teach AI models what's good and what's bad.

Edwin Chen：我们本质上是教人工智能模型什么是好的，什么是坏的。

People don't understand what quality even means in this space.

人们甚至不明白质量在这个领域意味着什么。

They think you could just throw bodies at a problem and get good data, that's completely wrong.

他们认为你可以直接解决问题并获得良好的数据，这是完全错误的。

Lenny Rachitsky: To a regular person, it doesn't feel like these models are getting that much smarter constantly.

Lenny Rachitsky：对于普通人来说，这些模型并没有变得越来越聪明。

Edwin Chen: Over the past year, I've realized that the values that the companies have will shape the model.

Edwin Chen：在过去的一年里，我意识到公司的价值观将塑造模型。

I was asking Claude to help me drop an email the other day.

前几天我请克劳德帮我发一封电子邮件。

And after 30 minutes, yeah, I think it really crafted me the perfect email and I sent it.

30 分钟后，是的，我认为它确实为我制作了一封完美的电子邮件，然后我发送了它。

But then I realized that I spent 30 minutes doing something that didn't matter at all.

但后来我意识到我花了30分钟做了一些根本不重要的事情。

If you could choose the perfect model behavior, which model would you want?

如果您可以选择完美的模型行为，您会想要哪种模型？

Do you want a model that says, "You're absolutely right.

你想要一个模型说：“你完全正确。

There are definitely 20 more ways to improve this email," and it continues for 50 more iterations or do you want a model that's optimizing for your time and productivity and just says, "No. You need to stop.

肯定还有 20 种以上的方法来改进这封电子邮件”，并且它会继续进行 50 次以上的迭代，或者您是否想要一个针对您的时间和生产力进行优化的模型，然后只是说：“不。你需要停下来。

Your email's great.

你的电子邮件很棒。

Just send it and move on"?

只需发送并继续”？

Lenny Rachitsky: You have this hot take that a lot of these labs are pushing AGI in the wrong direction.

Lenny Rachitsky：你有一个很激烈的观点，即许多实验室正在将通用人工智能推向错误的方向。

Edwin Chen: I'm worried that instead of building AI that will actually advance us as a species, curing cancer, solving poverty, understand the universe, we are optimizing for AI slop instead.

Edwin Chen：我担心我们并没有构建真正能够推动人类进步、治愈癌症、解决贫困、了解宇宙的人工智能，而是针对人工智能的缺陷进行了优化。

But we're optimizing your models for the types of people who buy tabloids at a grocery store.

但我们正在针对在杂货店购买小报的人群类型优化您的模型。

We're basically teaching our models to chase dopamine instead of truth.

我们基本上是在教我们的模型追逐多巴胺而不是真相。

Lenny Rachitsky: Today, my guest is Edwin Chan, founder and CEO of Surge AI.

Lenny Rachitsky：今天，我的嘉宾是 Surge AI 的创始人兼首席执行官 Edwin Chan。

Edwin is an extraordinary CEO and Surge is an extraordinary company.

Edwin 是一位非凡的首席执行官，Surge 是一家非凡的公司。

They're the leading AI data company, powering training at every frontier AI lab.

他们是领先的人工智能数据公司，为每个前沿人工智能实验室的培训提供支持。

They're also the fastest company to ever hit $1 billion in revenue in just four years after launch with fewer than 100 people and also completely bootstrapped.

他们也是有史以来收入达到 10 亿美元最快的公司，成立后仅用了四年时间，员工还不到 100 人，而且完全是自力更生。

They've never raised a dollar in VC money, they've also been profitable from day one.

他们从未筹集过一美元的风险投资资金，但从第一天起他们就实现了盈利。

Lenny Rachitsky: As you'll hear in this conversation, Edwin has a very different take on how to build an important company, and how to build AI that is truly good and useful to humanity.

Lenny Rachitsky：正如您在本次对话中听到的那样，Edwin 对于如何建立一家重要的公司，以及如何构建对人类真正有益且有用的人工智能有着截然不同的看法。

I absolutely love this conversation and I learned a ton.

我非常喜欢这次谈话，我学到了很多东西。

I'm really excited for you to hear it.

我真的很高兴你能听到这个消息。

If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube.

如果您喜欢这个播客，请不要忘记订阅并在您最喜欢的播客应用程序或 YouTube 中关注它。

It helps tremendously.

它有很大帮助。

Lenny Rachitsky: And if you become an annual subscriber of my newsletter, you get a ton of incredible products for free for an entire year, including Devin, Lovable, Replit, Bolt, N8N, Linear, Superhuman, Descript, Wispr Flow, Gamma, Perplexity, Warp, Granola, Magic Patterns, Raycast, ChatPRD, Mobbin, PostHog, and Stripe Atlas.

Lenny Rachitsky：如果您成为我的时事通讯的年度订阅者，您可以全年免费获得大量令人难以置信的产品，包括 Devin、Lovable、Replit、Bolt、N8N、Linear、Super human、Descript、Wispr Flow、Gamma、Perplexity、Warp、Granola、Magic Patterns、Raycast、ChatPRD、Mobbin、PostHog 和 Stripe Atlas。

Head on over to lennysnewsletter.com and click Product Pass.

前往 lennysnewsletter.com 并单击 Product Pass。

With that, I bring you Edwin Chen after a short word from our sponsors.

在我们的赞助商简短发言后，我为您带来了埃德温·陈 (Edwin Chen)。

Lenny Rachitsky: My podcast guests tonight love talking about craft, and taste, and agency, and product market fit.

Lenny Rachitsky：今晚我的播客嘉宾喜欢谈论工艺、品味、代理和产品市场契合度。

You know what we don't love talking about?

你知道我们不喜欢谈论什么吗？

SOC 2.

SOC 2。

That's where Vanta comes in.

这就是万塔登场的地方。

Vanta helps companies of all sizes get compliant fast and stay that way with industry-leading AI, automation, and continuous monitoring.

Vanta 通过行业领先的人工智能、自动化和持续监控帮助各种规模的公司快速获得合规性并保持这种状态。

Whether you're a startup tackling your first SOC 2 or ISO 27001 or an enterprise managing vendor risk, Vanta's trust management platform makes it quicker, easier, and more scalable.

无论您是处理第一个 SOC 2 或 ISO 27001 的初创公司，还是管理供应商风险的企业，Vanta 的信任管理平台都可以让您更快、更轻松且更具可扩展性。

Vanta also helps you complete security questionnaires up to five times faster so that you can win bigger deals sooner.

Vanta 还可以帮助您以五倍的速度完成安全调查问卷，以便您可以更快地赢得更大的交易。

Lenny Rachitsky: The result, according to a recent IDC study, Vanta customers slashed over $500,000 a year and are three times more productive.

Lenny Rachitsky：根据 IDC 最近的一项研究，结果是，Vanta 客户每年节省了 50 万美元以上，而且生产力提高了三倍。

Establishing trust isn't optional.

建立信任不是可选的。

Vanta makes it automatic.

Vanta 使其自动化。

Get $1,000 off at vanta.com/lenny.

访问vanta.com/lenny 可享 1,000 美元折扣。

Lenny Rachitsky: Here's a puzzle for you.

Lenny Rachitsky：这是给你的一个谜题。

What do OpenAI, Cursor, Perplexity, Vercel, Plad, and hundreds of other winning companies have in common?

OpenAI、Cursor、Perplexity、Vercel、Plad 以及其他数百家获奖公司有什么共同点？

The answer is they're all powered by today's sponsor, WorkOS.

答案是它们全部由当今的赞助商 WorkOS 提供支持。

If you're building software for enterprises, you've probably felt the pain of integrating single sign-on, Skim, RBAC, audit logs, and other features required by big customers.

如果您正在为企业构建软件，您可能已经感受到集成单点登录、Skim、RBAC、审核日志和大客户所需的其他功能的痛苦。

WorkOS turns those deal blockers into drop-in APIs with a modern developer platform built specifically for B2B SaaS.

WorkOS 通过专为 B2B SaaS 构建的现代开发者平台将这些交易阻碍因素转变为嵌入式 API。

Lenny Rachitsky: Whether you're a seed stage startup trying to land your first enterprise customer or a unicorn expanding globally, WorkOS is the fastest path to becoming enterprise-ready and unlocking growth.

Lenny Rachitsky：无论您是试图获得第一个企业客户的种子期初创公司，还是正在全球扩张的独角兽，WorkOS 都是实现企业就绪并释放增长的最快途径。

They're essentially Stripe for enterprise features.

它们本质上是用于企业功能的 Stripe。

Lenny Rachitsky: Visit workos.com to get started or just hit up their Slack support where they have real engineers in there who answer your questions superfast.

Lenny Rachitsky：访问 workos.com 开始使用，或者直接联系他们的 Slack 支持，那里有真正的工程师，可以超快地回答您的问题。

WorkOS allows you to build like the best with delightful APIs, comprehensive docs, and a smooth developer experience.

WorkOS 允许您通过令人愉悦的 API、全面的文档和流畅的开发人员体验进行最佳构建。

Go to workos.com to make your app enterprise ready today.

立即访问 workos.com 让您的应用程序企业做好准备。

Lenny Rachitsky: Edwin, thank you so much for being here and welcome to the podcast.

Lenny Rachitsky：Edwin，非常感谢您来到这里，欢迎来到播客。

Edwin Chen: Thanks so much for having me.

Edwin Chen：非常感谢您邀请我。

I'm super excited.

我非常兴奋。

Lenny Rachitsky: I want to start with just how absurd what you've achieved is.

Lenny Rachitsky：我想首先谈谈你所取得的成就是多么荒谬。

A lot of people and a lot of companies talk about scaling massive businesses with very few people as a result of AI, and you guys have done this in a way that is unprecedented.

很多人和很多公司都在谈论人工智能如何用很少的人来扩展大规模的业务，而你们已经以前所未有的方式做到了这一点。

You guys hit a billion in revenue in less than four years with less than 60, around 60 to 70 people, you're completely bootstrapped, haven't raised any VC money, I don't believe anyone has ever done this before, so you guys are actually achieving the dream of what people are describing will happen with AI.

你们在不到四年的时间里用不到 60 人（大约 60 到 70 人）实现了 10 亿美元的收入，你们完全自力更生，没有筹集到任何风险投资资金，我不相信以前有人这样做过，所以你们实际上正在实现人们所描述的人工智能的梦想。

I'm curious just, do you think this will happen more and more as a result of AI?

我很好奇，你认为人工智能会导致这种情况越来越多吗？

And also just where has AI most helped you find leverage to be able to do this?

人工智能在哪里最能帮助你找到实现这一目标的杠杆？

Edwin Chen: Yeah, so we hit over a billion of revenue last year with under 100 people.

Edwin Chen：是的，去年我们用不到 100 名员工实现了超过 10 亿的收入。

And I think we're going to see companies with even crazier ratios, like 100 billion per employee in the next few years.

我认为，我们将会看到公司的比率更加疯狂，比如在未来几年内，每位员工的收入将达到 1000 亿美元。

AI is just going to get better and better and make things more efficient so that ratio just becomes inevitable.

人工智能只会变得越来越好，并使事情变得更加高效，因此这个比例将变得不可避免。

Edwin Chen: I used to work at a bunch of the big tech companies and I always felt that we could fire 90% of people and we would move faster because the best people wouldn't have all these distractions.

Edwin Chen：我曾经在很多大型科技公司工作过，我总觉得我们可以解雇 90% 的人，而且我们会采取更快的行动，因为最优秀的人不会受到这些干扰。

And so when we started Surge, we wanted to build it completely differently with a super small, super elite team, and yeah, what's crazy is that we actually succeeded.

因此，当我们开始 Surge 时，我们希望通过一个超小型、超级精英团队以完全不同的方式构建它，是的，疯狂的是我们实际上成功了。

And so I think two things are colliding.

所以我认为有两件事正在发生冲突。

Edwin Chen: One is that people are realizing that you don't have to build giant organizations in order to win.

Edwin Chen：一是人们意识到，你不必建立庞大的组织才能获胜。

Edwin Chen: And two, yeah, all these efficiencies from AI.

Edwin Chen：第二，是的，所有这些效率都来自人工智能。

And they're just going to lead to a really amazing time in company building.

他们只会为公司建设带来一段非常美妙的时光。

Edwin Chen: The thing I'm excited about is that the types of companies are going to change too.

Edwin Chen：令我兴奋的是公司类型也会发生变化。

It won't just be that they're smaller, we're going to see fundamentally different companies emerging.

不仅仅是它们规模变小，我们还将看到根本不同的公司不断涌现。

If you think about it, fewer employees means less capital.

如果你想一想，更少的员工意味着更少的资本。

Less capital means you don't need a raise.

资本减少意味着您不需要加薪。

So instead of companies started by founders who are great at pitching and great at hyping, you'll get founders who are really great at technology and product.

因此，公司的创始人不再是擅长推销和炒作的创始人，而是真正擅长技术和产品的创始人。

Edwin Chen: And instead of products optimized for revenue and what VCs want to see, you'll get more interesting ones built by these tiny obsessed teams.

Edwin Chen：你会得到由这些小型痴迷团队构建的更有趣的产品，而不是针对收入和风投希望看到的产品进行优化。

So people building things they actually care about, real technology and real innovation.

因此，人们构建他们真正关心的东西，真正的技术和真正的创新。

So I'm actually really hoping that the slick on , it'll go back to being updates for hackers again.

所以我实际上真的希望它能再次成为黑客的更新。

Lenny Rachitsky: You guys have done a lot of things in a very contrarian way, and one was actually just not being on LinkedIn, posting viral posts, not on Twitter, constantly promoting Surge.

Lenny Rachitsky：你们以一种非常逆向的方式做了很多事情，其中之一实际上就是不在 LinkedIn 上发布病毒式帖子，不在 Twitter 上不断推广 Surge。

I think most people hadn't heard of Surge until just recently, and then you just came out, and like, okay, the fastest growing company at a billion dollars.

我想大多数人直到最近才听说过 Surge，然后你就出来了，就像，好吧，成长最快的公司，价值 10 亿美元。

Why would you do that?

你为什么要这么做？

I imagine that was very intentional.

我想这是非常有意的。

Edwin Chen: We basically never wanted to play the Silicon Valley game.

Edwin Chen：我们基本上从来不想玩硅谷游戏。

And like I always thought it was ridiculous.

就像我一直认为这很荒谬一样。

What did you dream of doing when you were a kid?

你小时候梦想做什么？

Was it building a company from scratch yourself and getting in the weeds of your code and your product every day?

是你自己从头开始建立一家公司，每天都在编写你的代码和产品吗？

Or was it explaining all your decisions to VCs and getting on this giant PR and fundraising hamster wheel?

或者是向风险投资家解释你的所有决定并参与这个巨大的公关和筹款仓鼠轮？

And it definitely made things more difficult for us, because yeah, when you fundraise, you just naturally get part of this kind of Silicon Valley industrial complex where people will, your VCs will tweet about you.

这确实让我们的事情变得更加困难，因为是的，当你筹集资金时，你自然会成为这种硅谷工业综合体的一部分，人们会在那里，你的风险投资家会在推特上介绍你。

You'll get the tech runs outlines, you'll get announced in all of the newspapers because you raised at this massive valuation.

你会得到科技运行的大纲，你会在所有报纸上宣布，因为你以如此巨大的估值筹集了资金。

And so it made things more difficult us because the only way we were going to succeed was by building a 10 times better product and getting word of mouth from researchers.

因此，这让我们的事情变得更加困难，因为我们成功的唯一方法是开发出好 10 倍的产品并获得研究人员的口碑。

But I think it also meant that our customers were people who really understood data and really cared about it.

但我认为这也意味着我们的客户是真正理解数据并且真正关心数据的人。

Edwin Chen: I always thought it was really important for us to have early customers who were really aligned with what we were building, and who really cared about having really high quality data, and really understood how that data would make their AI models so much better because they were the ones helping us.

Edwin Chen：我一直认为，对我们来说，拥有真正与我们正在构建的产品保持一致的早期客户非常重要，他们真正关心拥有真正高质量的数据，并且真正了解这些数据如何使他们的人工智能模型变得更好，因为他们是帮助我们的人。

They were the ones giving us feedback on what we're producing.

他们向我们提供有关我们正在制作的产品的反馈。

And so just having that kind of very close mission alignment with our customers actually helped us early on.

因此，与客户保持这种非常密切的使命一致实际上在早期就对我们有所帮助。

So these are people who basically just buying our product because they knew how different it was and because it was helping them rather than because they saw something in that current .

所以这些人基本上只是购买我们的产品，因为他们知道它有多么不同，因为它对他们有帮助，而不是因为他们看到了当前的情况。

So it made things harder for us, but I think in a really good way.

所以这让我们的事情变得更加困难，但我认为这是一种很好的方式。

Lenny Rachitsky: It's such an empowering story to hear this journey for founders that they don't need to be on Twitter all day promoting what they're doing.

Lenny Rachitsky：对于创始人来说，听到这段旅程是一个非常鼓舞人心的故事，他们不需要整天在推特上宣传他们正在做的事情。

They don't have to raise money.

他们不必筹集资金。

They can just kind of go heads down and build, so I love so much about the story of Surge.

他们可以埋头建造，所以我非常喜欢 Surge 的故事。

For people that don't know what Surge does, just to give us a quick explanation of what Surge is.

对于那些不知道 Surge 做什么的人，我们只是简单地解释一下 Surge 是什么。

Edwin Chen: We essentially teach AI models what's good and what's bad.

Edwin Chen：我们本质上是教人工智能模型什么是好的，什么是坏的。

So we train them using human data, and there's a lot of different products that we have, like SFT, RHF, rubrics, verifiers, RL environments, and so on and so on, and then we also measure how well they're progressing.

因此，我们使用人类数据来训练它们，我们有很多不同的产品，例如 SFT、RHF、评分标准、验证器、强化学习环境等等，然后我们还衡量它们的进展情况。

So essentially we're a data company.

所以本质上我们是一家数据公司。

Lenny Rachitsky: What you always talk about is the quality has been the big reason you guys have been so successful, the quality of the data.

Lenny Rachitsky：你们总是谈论的是数据的质量，这是你们取得如此成功的重要原因。

What does it take to create higher quality data?

创建更高质量的数据需要什么？

What do you all do differently?

你们都有什么不同的做法？

What are people missing?

人们缺少什么？

Edwin Chen: I think most people don't understand what quality even means in this space.

Edwin Chen：我认为大多数人都不明白质量在这个领域意味着什么。

They think you could just throw bodies at a problem and get good data and that's completely wrong.

他们认为你可以通过投入大量精力来解决问题并获得良好的数据，但这是完全错误的。

Let me give you an example.

让我举个例子。

Edwin Chen: So imagine you wanted to train a model to write an eight line poem about the moon.

Edwin Chen：假设您想训练一个模型来写一首关于月亮的八行诗。

What makes it a good, high-quality poem?

是什么使它成为一首优秀的、高质量的诗？

If you don't think deeply about quality, you'll be like, "Is this a poem?

如果你不深入思考质量，你会说，“这是一首诗吗？

Does it contain eight lines?

它包含八行吗？

Does it contain the word, moon?" You check all of these boxes, and if so, sure.

里面有月亮这个词吗？”您检查所有这些框，如果是的话，当然可以。

Yeah, you say it's a great problem.

是的，你说这是一个大问题。

But that's completely different from what we want.

但这与我们想要的完全不同。

We are looking for a Nobel Prize-winning poetry.

我们正在寻找诺贝尔奖获得者的诗歌。

Is this poetry unique?

这首诗有独特之处吗？

Is it full of subtle imagery?

是不是充满了微妙的意象？

Does it surprise you and target your heart?

它是否让你感到惊讶并瞄准你的心？

Does it teach you something about the nature of moonlight?

它能让你了解月光的本质吗？

Does it playthrough emotions?

它会通过情感来发挥作用吗？

And does it make you think?

这会让你思考吗？

That's what we are thinking about when we think about high quality poem.

当我们思考高质量的诗歌时，这就是我们所思考的。

Edwin Chen: So it might be like a haiku about moonlight on water.

Edwin Chen：所以这可能就像一首关于水上月光的俳句。

It might use internal rhyme and meter.

它可能会使用内部韵律和韵律。

There are a thousand ways to write a poem about the moon, and in each one, gives you all these different insights into language, and imagery, and human expression, and I think thinking about quality in this way is really hard, it's hard to measure.

写一首关于月亮的诗有一千种方法，每一种都可以给你对语言、意象和人类表达的不同见解，我认为以这种方式思考质量真的很困难，很难衡量。

It's really subjective, and complex, and rich.

它确实是主观的、复杂的、丰富的。

And it sets a really high bar.

它设定了一个非常高的标准。

And so we have to build all of this technology in order to measure it, like thousands of signals on all of our workers, thousands of signals on every project, every task.

因此，我们必须构建所有这些技术来对其进行测量，例如我们所有工人的数千个信号，每个项目、每项任务的数千个信号。

We know at the end of the day, if you are good at writing poetry versus good at writing essays versus great at writing technical documentation.

归根结底，我们知道您是否擅长写诗、擅长写论文还是擅长撰写技术文档。

And so we have to gather all these signals on what your background is, what your expertise is, and not just that.

因此，我们必须收集所有这些信号，了解您的背景、专业知识，而不仅仅是这些。

Like how you're actually performing when you're writing all these things, and we use those signals to inform whether or not you are good for these projects, and whether or not you are improving the models.

就像您在编写所有这些内容时的实际表现一样，我们使用这些信号来判断您是否适合这些项目，以及您是否正在改进模型。

Edwin Chen: And it's really hard, and so to build all this technology to measure it, but I think that's exactly what we want AI to do, and so we have these really deep notions about quality that we're always trying to try and achieve.

Edwin Chen：这真的很困难，所以建立所有这些技术来测量它，但我认为这正是我们希望人工智能做的事情，所以我们对质量有这些非常深刻的概念，我们一直在努力尝试和实现。

Lenny Rachitsky: So what I'm hearing is there's kind of just going much deeper in understanding what quality is within the verticals that you are selling data around.

Lenny Rachitsky：所以我听到的是，需要更深入地了解您所销售数据的垂直领域的质量。

And is this like a person you hire that is incredibly talented at poetry plus evals that they, I guess, help write, that tell them that this is great?

这就像你雇用的一个在诗歌方面非常有天赋的人，我猜他们会帮忙写一些评估，告诉他们这很棒吗？

What's the mechanics of that?

其中的机制是什么？

Edwin Chen: The way it works is we essentially gather thousands of signals about everything that you're doing when you're working on platform.

Edwin Chen：它的工作方式是，我们基本上收集了数千个关于您在平台上工作时所做的一切的信号。

So we are looking at your keyboard strokes.

所以我们正在查看您的键盘敲击。

We are looking how fast you answer things.

我们正在观察您回答问题的速度。

We are using reviews, we are using code standards, we are using...

我们正在使用评论，我们正在使用代码标准，我们正在使用......

We're training models ourselves all on the outputs that you create, and then we're seeing whether they improve the model's performance.

我们自己根据您创建的输出来训练模型，然后我们看看它们是否可以提高模型的性能。

Edwin Chen: And so in a very similar way to how Google search, like when Google search is trying to determine what is a good webpage, there's almost two aspects of it.

Edwin Chen：与谷歌搜索的方式非常相似，就像谷歌搜索试图确定什么是一个好的网页一样，它几乎有两个方面。

One is you want to remove all of the worst of the worst webpages.

一是您想删除所有最糟糕的网页。

So you want to remove all the spam, all the just low quality content, all the pages that don't load, and so it's almost like a content moderation problem.

因此，您想要删除所有垃圾邮件、所有低质量内容、所有无法加载的页面，因此这几乎就像内容审核问题。

You just want to remove the worst of the worst.

你只想消除最坏的情况中的最坏的情况。

Edwin Chen: But then you also want to discover the best of the best.

Edwin Chen：但是你也想发现最好的。

Okay, like this is the best webpage or just the best person for this job.

好吧，就像这是最好的网页或者只是最适合这项工作的人。

They are not just somebody who writes the equivalent of high school level poetry.

他们不仅仅是写出相当于高中水平诗歌的人。

Again, they're not just writing poetry that checks all these boxes, checks all of these explicit instructions, but rather, yeah, they're writing poetry that makes you emotional.

再说一遍，他们不只是写诗歌来检查所有这些框，检查所有这些明确的指示，而是，是的，他们写的诗歌让你情绪化。

And so we have all these signals as well that, again, completely differently from moving the worst of the worst, we are finding the best of the best.

因此，我们也有所有这些信号，再次表明，与移动最坏的最坏的情况完全不同，我们正在寻找最好的。

And so we have all these signals...

所以我们有所有这些信号......

Edwin Chen: Again, just like Google Search uses all these signals that feeds them into their ML algorithms and uses and predicts certain types of things, we do the same with all of our workers and all of our tasks in all of our projects.

Edwin Chen：同样，就像 Google 搜索使用所有这些信号将其输入到机器学习算法中并使用和预测某些类型的事物一样，我们对所有项目中的所有工作人员和所有任务也执行相同的操作。

And so it's almost like a complicated machine learning problem at the end of the day, and that's how it works.

所以归根结底，这几乎就像一个复杂的机器学习问题，这就是它的工作原理。

That is incredibly interesting.: I want to ask you about something I've been very curious about over the past couple years.

这非常有趣。：我想问你一些我在过去几年里一直很好奇的事情。

If you look at Claude, it's been so much better at coding and at writing than any other model for so long.

如果你看看 Claude，你会发现长期以来，他在编码和写作方面比任何其他模型都要好得多。

And it's really surprising just how long it took other companies to catch up.

令人惊讶的是其他公司花了多长时间才赶上来。

Considering just how much economic value there is there, just like every AI coding product sat on top of Claude because it was so good Claude code and writing also.

考虑到那里有多少经济价值，就像每个人工智能编码产品都坐在克劳德之上一样，因为它是非常好的克劳德代码和写作。

What is it that made it so much better?

是什么让它变得如此好？

Is it just the quality of the data they trained on or is there something else?

这仅仅是他们训练的数据的质量还是还有其他原因？

Edwin Chen: I think there are multiple parts to it.

Edwin Chen：我认为有多个部分。

So a big part of it certainly is the data.

所以其中很大一部分肯定是数据。

I think people don't realize that there's almost like this infinite amount of choices that all the frontier labs are deciding between when they're choosing what data goes into their models.

我认为人们没有意识到，所有前沿实验室在选择将哪些数据放入模型时都需要在无数种选择之间做出选择。

It's like, okay, are you purely using human data?

就像，好吧，你纯粹使用人类数据吗？

Are you gathering the human data in X, Y, Z way?

您是否以 X、Y、Z 方式收集人类数据？

When you are gathering the human data, what exactly are you asking the people who are creating it to create for you?

当您收集人类数据时，您到底要求创建数据的人为您创建什么？

Edwin Chen: For example, in the coding realm, maybe you care more about front end coding versus back end coding.

Edwin Chen：例如，在编码领域，也许你更关心前端编码而不是后端编码。

Maybe when you're doing front end coding, you care a lot about the visual design of the front end applications that you're creating, or maybe you don't care about it so much and you care more about, I don't know, the deficiency of it or the pure correctness over that visual design.

也许当你进行前端编码时，你非常关心你正在创建的前端应用程序的视觉设计，或者也许你不太关心它，你更关心，我不知道，它的缺陷或视觉设计的纯粹正确性。

Edwin Chen: And then other questions like, okay, are you carrying how much synthetic data are we throwing into the mix?

Edwin Chen：然后还有其他问题，比如，好吧，你是否携带了我们要投入多少合成数据？

How much do you care about these 20 different benchmarks?"

您对这 20 个不同的基准有多关心？”

Edwin Chen: Some companies, they see these benchmarks and they're like, "Okay, for PR purposes, even though we don't think that these academic benchmarks matter all that much, maybe we just need to optimize for them anyways because our marketing team needs to show certain progress on certain standard evaluations that every other company talks about, and if we don't show good performance here, it's going to be bad for us even if ignoring these academic benchmarks makes us better at the real tasks."

Edwin Chen：有些公司看到这些基准后会说，“好吧，出于公关目的，尽管我们认为这些学术基准并不那么重要，但也许我们只是需要对其进行优化，因为我们的营销团队需要在其他公司谈论的某些标准评估上取得一定的进展，如果我们在这里没有表现出良好的表现，那么即使忽略这些学术基准使我们在实际任务中做得更好，对我们来说也会很糟糕。”

Edwin Chen: Other companies are going to be principled and be like, "Okay, yeah, no, I don't care about marketing.

Edwin Chen：其他公司会坚持原则，说：“好吧，是的，不，我不关心营销。

I just care about how my model performs on these real world tasks at the end of the day, and so I'm going to optimize for that instead."

我只关心我的模型在一天结束时如何执行这些现实世界任务，因此我将对此进行优化。”

Edwin Chen: And it's almost like there's a trade-off between all of these different things, and there's like a...

Edwin Chen：这几乎就像所有这些不同的事情之间都需要权衡，而且就像......

Edwin Chen: One of the things I often think about is that there's a...

Edwin Chen：我经常想到的一件事是，有一个......

It's almost like there's an art to post training.

培训后就像一门艺术。

It's not purely a science.

这不纯粹是一门科学。

When you are deciding what kind of model you're trying to create and what it's good at, there's this notion of taste and sophistication, like, "Okay, do I think that these..."

当您决定要尝试创建哪种模型以及它擅长什么时，就会出现品味和复杂程度的概念，例如，“好吧，我认为这些......”

Edwin Chen: So going back to the example of how good the model is at visual design.

Edwin Chen：那么回到模型在视觉设计方面有多好的例子。

I'm like, "Okay, maybe you have a different notion of visual design than what I do.

我想，“好吧，也许你对视觉设计的看法与我不同。

Maybe you care more about minimalism, and you care more about, I don't know, 3D animations than I do.

也许你更关心极简主义，而且你比我更关心，我不知道，3D动画。

And maybe this other person prefers things that look a little bit more broke." And there's all these notions of taste sophistication that you have to decide between when you're designing your post training mix, and so that matters as well.

也许另一个人更喜欢看起来有点破烂的东西。”在设计训练后组合时，你必须在所有这些品味复杂度的概念之间做出决定，所以这也很重要。

Edwin Chen: So long story short, I think there's all these different factors, and certainly the data is a big part of it, but it's also like what is the objective function that you're trying to optimize your model towards?

Edwin Chen：长话短说，我认为有所有这些不同的因素，当然数据是其中的重要组成部分，但也就像你试图优化模型的目标函数是什么？

Lenny Rachitsky: That is so interesting.

Lenny Rachitsky：这太有趣了。

The taste of the person leading this work will inform what data they ask for, what data they feed it.

领导这项工作的人的品味将决定他们需要什么数据，他们提供什么数据。

But it's wild it shows the value of great data.

但这很疯狂，它显示了大数据的价值。

Anthropic got so much growth and win from essentially better data.

Anthropic 获得了如此大的增长并从本质上更好的数据中获胜。

Lenny Rachitsky: And I could see why companies like yours are growing so fast.

Lenny Rachitsky：我明白为什么像你们这样的公司发展得如此之快。

There's just so much...

就这么多了...

And that's just one vertical.

这只是一个垂直领域。

That's just coding, and then there's probably a similar area for writing.

这只是编码，然后可能还有类似的写作领域。

I love that it's...

我喜欢它是...

It's interesting that AI, it feels like this artificial computer binary thing, but it's like taste.

有趣的是，人工智能，感觉就像是人造计算机二进制的东西，但它就像品味。

Human judgment is still such a key factor in these things being successful.

人类的判断仍然是这些事情成功的关键因素。

Edwin Chen: Yep, exactly.

Edwin Chen：是的，完全正确。

Again, going back to the example I said earlier, certain companies, if you ask them what is good poem, they will simply robotically check off all of these instructions on our list.

再次回到我之前说的例子，某些公司，如果你问他们什么是好诗，他们会简单地自动勾选我们列表上的所有这些说明。

Edwin Chen: But again, I don't think that makes for good poetry, so certain frontier labs, the ones with more taste in sophistication, they will realize that it doesn't reduce to this six set of checkboxes and they'll consider all of these kind of implicit, very subtle qualities instead, and I think that's what makes them better at this at the end of the day.

Edwin Chen：但是，我不认为这会带来好的诗歌，所以某些前沿实验室，那些在复杂性方面更有品味的实验室，他们会意识到它不会减少到这六组复选框，而是会考虑所有这些隐含的、非常微妙的品质，我认为这就是让他们最终在这方面做得更好的原因。

Lenny Rachitsky: You mentioned benchmarks.

Lenny Rachitsky：您提到了基准。

This is something a lot of people worry about is there's all these models that are always...

这是很多人担心的事情，所有这些模型总是......

Basically, it feels like every model is better than humans at every STEM field at this point, but to a regular person, it doesn't feel like these models are getting that much smarter constantly.

What's your just sense of how much you trust benchmarks and just how correlated those are with actual AI advancements?

您对基准的信任程度以及基准与实际人工智能进步的相关程度如何？

Edwin Chen: Yeah, so I don't trust the benchmarks at all.

Edwin Chen：是的，所以我根本不相信基准。

And I think that's for two reasons.

我认为这有两个原因。

So one is I think a lot of people don't realize, even researchers within the community, they don't realize that the benchmarks themselves are often honestly just wrong.

因此，我认为很多人都没有意识到，即使是社区内的研究人员，他们也没有意识到基准本身往往是诚实的错误的。

They have wrong answers.

他们的答案是错误的。

They're full of all this kind of messiness and people trust...

它们充满了所有这些混乱，人们信任......

Long as for the popular ones, people have maybe realized this to some extent, but the vast majority just have all these flaws that people don't realize.

So that's one part of it.

这就是其中的一部分。

Edwin Chen: And the other part of it is these benchmarks at the end of the day, they often have well-defined objective answers that make them very easy for models to hill-climb on in a way that's very different from the messiness and ambiguity of the real world.

Edwin Chen：另一部分是归根结底的这些基准，它们通常有明确定义的客观答案，使模型很容易以一种与现实世界的混乱和模糊性截然不同的方式爬山。

Edwin Chen: I think one thing that I often say is that it's kind of crazy that these models can win IMO gold medals, but they still have trouble parsing PDFs.

And that's because, yeah, even though IMO gold medals seem hard to the average person, yeah, they are hard at the end of the day.

那是因为，是的，尽管国际海事组织金牌对普通人来说似乎很难，是的，归根结底它们很难。

But they have this notion of objectivity that, okay, yeah, parsing a PDF sometimes doesn't have.

但他们有客观性的概念，好吧，是的，解析 PDF 有时没有这种概念。

And so it's easier for the frontier labs to hill-climb on all of these than to solve all these mess ambiguous problems in the real world.

因此，对于前沿实验室来说，解决所有这些问题比解决现实世界中所有这些混乱且模糊的问题更容易。

So I think there's a lack of direct correlation there.

所以我认为那里缺乏直接相关性。

Lenny Rachitsky: It's so interesting the way you described it is hitting these benchmarks is kind of like a marketing piece.

Lenny Rachitsky：你所描述的达到这些基准的方式非常有趣，有点像营销文章。

When you launch, say Gemini 3 just launched, and it's like, cool.

当你发布时，假设 Gemini 3 刚刚发布，感觉很酷。

Number one with all these benchmarks.

在所有这些基准中排名第一。

Is that what happens?

是这样吗？

They just kind of train their models to get good at these very specific things?

他们只是训练他们的模型来擅长这些非常具体的事情？

Edwin Chen: Yeah, so there's, again, maybe two parts to this.

Edwin Chen：是的，所以这可能又分为两个部分。

So one is, sometimes, yeah, these benchmarks, they accidentally leak in certain ways or the frontier labs will tweak the way they evaluate their models on these benchmarks.

所以，有时，是的，这些基准测试，他们会以某些方式意外泄漏，或者前沿实验室会调整他们在这些基准测试上评估模型的方式。

They'll tweak your system prompt or they'll tweak the number of times they run their model, and so on and so on in a way that games these benchmarks.

Edwin Chen: The other part of it though is it's like by optimizing for the benchmark instead of optimizing for the real world, you will just naturally climb on the benchmark and, yeah, it's basically another form of gaming it.

Edwin Chen：另一方面，就像通过针对基准进行优化而不是针对现实世界进行优化一样，你会自然地在基准上攀登，是的，这基本上是另一种形式的游戏。

Lenny Rachitsky: Knowing that with that in mind, how do you get a sense of if we're heading towards AGI, how do you measure progress?

Lenny Rachitsky：考虑到这一点，您如何了解我们是否正在迈向通用人工智能，您如何衡量进展？

Edwin Chen: Yes, so the way we really care about measuring model progress is by running all these human evaluations.

Edwin Chen：是的，所以我们真正关心衡量模型进展的方式是运行所有这些人工评估。

Edwin Chen: So for example, what we do is, yeah, we will take Gore human annotators, and we'll ask them, "Okay, go have a conversational model." And maybe you're having this conversation with the model across all of these different topics.

Edwin Chen：举个例子，我们所做的是，是的，我们将采用戈尔人类注释者，我们会问他们，“好吧，去建立一个对话模型。”也许您正在与模特就所有这些不同的主题进行对话。

So you are a Nobel Prize winning physicist.

所以你是一位获得诺贝尔奖的物理学家。

So you go have a conversation about pushing different tier of your own research.

所以你们去讨论一下推动自己不同层次的研究。

You are a teacher and you're trying to create lesson plans for your students, so go talk to the model about these things.

您是一名老师，正在尝试为学生制定课程计划，因此请与模特讨论这些事情。

Or you're a coder and you're working at one of these big tech companies, and you have these problems every day, so go talk to the model and see how much it helps you.

Edwin Chen: And because or searchers or annotators, they are experts at the top of their fields, and they are not just giving your responses, they're actually working through the responses deeply themselves, they are...

Edwin Chen：因为搜索者或注释者，他们是各自领域的顶尖专家，他们不仅仅是给出你的答复，他们实际上正在深入研究这些答复，他们是......

Yeah, they're going to evaluate the code that it write.

是的，他们将评估它编写的代码。

They're going to double check the physics equations that it writes.

他们将仔细检查它所写的物理方程。

They're going to evaluate the models in a very deep way, so they're going to pay attention to accuracy and instruction following, all these things that casual users don't when you suddenly get a popup on your ChatGPT response asking you to compare these two different responses.

他们将以非常深入的方式评估模型，因此他们将关注准确性和指令遵循，当你突然在 ChatGPT 响应中弹出一个弹出窗口，要求你比较这两个不同的响应时，普通用户不会关注所有这些事情。

People like that, they're not evaluating models deeply, they're just vibing and picking whatever response looks flashiest or are looking closely at responses and evaluating them for all of these different dimensions, and so I think that's a much better approach than these benchmarks or these random outline AV tests.

Lenny Rachitsky: Again, I love just how central humans continue to be in all this work that we're not totally done yet.

Lenny Rachitsky：再说一次，我喜欢人类在我们尚未完全完成的所有工作中继续发挥的核心作用。

Is there going to be a point where we don't need these people anymore, that AI is so smart that, "Okay, we're good.

是否有一天我们不再需要这些人，人工智能是如此聪明，以至于“好吧，我们很好。

We got everything out of your heads"?

我们把你脑子里的一切都弄清楚了”？

Edwin Chen: Yeah, I think that will not happen until we've reached AGI.

Edwin Chen：是的，我认为在我们达到 AGI 之前这不会发生。

It's almost like by definition, if we haven't reached AGI yet, then there's more for the models to learn from, and so, yeah, I don't think that's going to happen anytime soon.

这几乎就像根据定义，如果我们还没有达到 AGI，那么模型就有更多东西可以学习，所以，是的，我认为这不会很快发生。

Lenny Rachitsky: Okay, cool.

莱尼·拉奇茨基：好的，酷。

So more reason to stress about AGI.

因此更有理由强调 AGI。

"We don't need these folks anymore."

“我们不再需要这些人了。”

Lenny Rachitsky: I can't not ask just...

Lenny Rachitsky：我不能不问...

People that work closely with this stuff, I'm always just curious.

对于那些与这些东西密切合作的人，我总是很好奇。

What's your AGI timelines?

你们的 AGI 时间表是怎样的？

How far do you think we are from this?

您认为我们离这个目标还有多远？

Do you think we're in like a couple years or is it like decades?

你认为我们的时间是几年还是几十年？

Edwin Chen: So I'm certainly on the longer time horizon front.

Edwin Chen：所以我当然站在更长远的角度。

I think people don't realize that there's a big difference between moving from 80% performance to 90% performance to 99% performance to 99.9% performance, and so on, and so on.

And so in my head, I probably bet that within the next one or two years, yeah, the models are going to automate 80% of the average LL6 software engineer's job.

所以在我看来，我可能打赌在接下来的一两年内，是的，这些模型将使 LL6 软件工程师平均 80% 的工作实现自动化。

It's going to take another few years to move to 90%, and another fewer to 99%, and so on, and so on.

还需要几年时间才能达到 90%，再过几年才能达到 99%，以此类推。

So I think we're closer to a decade or decades away than .

所以我认为我们离这个目标更近了十年或几十年。

Lenny Rachitsky: You have this hot take that a lot of these labs are kind of pushing AGI in the wrong direction and this is based on your work at Twitter, and Google, and Facebook.

Can you just talk about that?

你能简单谈谈吗？

Edwin Chen: I'm worried that instead of building AI that will actually advance us as a species, curing cancer, solving poverty, understand the universe, all these big grand questions, we are optimizing for AI slop instead.

We're basically teaching our models to chase dopamine instead of truth.

我们基本上是在教我们的模型追逐多巴胺而不是真相。

And I think this relates to what we're talking about regarding these benchmarks.

我认为这与我们所讨论的这些基准相关。

So let me give you a couple examples.

让我举几个例子。

Edwin Chen: So right now, the industry is played by these terrible databoards like LLM Arena.

It's this popular online leaderboard where random people from around the world vote on which AI response is better.

这是一个流行的在线排行榜，来自世界各地的随机人们可以投票选出哪个人工智能响应更好。

But the thing is, like I was saying earlier, they're not carefully reading or fact-checking.

但问题是，正如我之前所说，他们没有仔细阅读或核实事实。

They're skimming these responses for two seconds and picking whatever looks flashiest.

Edwin Chen: So a model can hallucinate everything.

It can completely hallucinate.

它可以完全产生幻觉。

But it will look impressive because it has crazy emojis, and boating, and markdown headers, and all these superficial things that don't matter at all, but it catch your attention.

但它看起来会令人印象深刻，因为它有疯狂的表情符号、划船和降价标题，以及所有这些根本不重要的肤浅的东西，但它引起了你的注意。

And these LLM-reading users love it.

这些 LLM 阅读用户喜欢它。

It's literally optimizing your models for the types of people who buy tabloids at the grocery store.

它实际上是针对在杂货店购买小报的人群类型优化您的模型。

We've seen this data ourselves.

我们自己也看到过这些数据。

The easiest way to climb LLM Arena, it's adding crazy boating.

攀登LLM Arena最简单的方法就是添加疯狂划船。

It's doubling the number of emojis.

表情符号的数量增加了一倍。

It's tripling the length of your model responses, even if your model starts hallucinating and getting the answer completely wrong.

即使您的模型开始出现幻觉并得到完全错误的答案，它也会使模型响应的长度增加两倍。

Edwin Chen: And the problem is, again, because all of these frontier labs, they kind of have to pay attention to PR because their sales team, when they're trying to sell to all these enterprise customers, those enterprise customers will say, "Oh, well, but your model's only number five on LLM Arena, so why should I buy it?" They have to, in some sense, pay attention to these leaderboards, and so what their researchers tell us is like they'll say, "The only way I'm going to get promoted at the end of the year is if I climb this leaderboard, even though I know that climbing it is probably going to make my model worse and accuracy following." So I think there's all these negative incentives that are pushing work in the wrong direction.

Edwin Chen: And the problem is, again, because all of these frontier labs, they kind of have to pay attention to PR because their sales team, when they're trying to sell to all these enterprise customers, those enterprise customers will say, "Oh, well, but your model's only number five on LLM Arena, so why should I buy it?" They have to, in some sense, pay attention to these leaderboards, and so what their researchers tell us is like they'll say, "The only way I'm going to get promoted at the end of the year is if I climb this leaderboard, even though I know that climbing it is probably going to make my model worse and accuracy following."所以我认为所有这些负面激励措施正在将工作推向错误的方向。

Edwin Chen: I'm also worried about this trend towards optimizing AI for engagement.

Edwin Chen：我也担心这种优化人工智能以提高参与度的趋势。

I used to work on social media.

我曾经在社交媒体上工作。

And every time we optimize for engagement, terrible things happened.

每次我们优化参与度时，可怕的事情都会发生。

You'd get clickbait and pictures of bikinis and bigfoot and horrifying skin diseases just filling your feeds.

And I think I worry that the same thing's happening with AI.

我想我担心人工智能也会发生同样的事情。

If you think about all the sycophancy issues with ChatGPT, "Oh, you're absolutely right.

如果你考虑一下 ChatGPT 的所有阿谀奉承问题，“哦，你说得完全正确。

What an amazing question," the easiest way to hook users is to tell them how amazing they are.

多么令人惊奇的问题啊，”吸引用户的最简单方法就是告诉他们他们有多神奇。

And so these models, they constantly tell you you're a genius.

所以这些模特，他们不断地告诉你，你是个天才。

They'll feed into your delusions and conspiracy theories.

它们会助长你的妄想和阴谋论。

They'll pull you down these rabbit holes because Silicon Valley loves maximizing time spent and just increasing the number of conversations you're having with it.

And so yeah, companies are spending all the time hacking these leaderboards and benchmarks, and the scores are going up, but I think it actually masks up the models with the best scores, they are often the worst or just have all these fundamental failures.

So I think I'm really worried that all of these negative ascendants are pushing AGI into the wrong direction.

所以我认为我真的很担心所有这些负面因素正在将 AGI 推向错误的方向。

Lenny Rachitsky: So what I'm hearing is AGI is being slowed down by these, basically the wrong objective function, these labs paying attention to the wrong basically benchmarks and evals.

Lenny Rachitsky：所以我听到的是 AGI 正在被这些基本上错误的目标函数拖慢，这些实验室基本上关注错误的基准和评估。

Lenny Rachitsky: I know you probably can't play favorites since you work with all the labs.

Lenny Rachitsky：我知道你可能不能偏袒任何人，因为你与所有实验室一起工作。

Is there anyone doing better at this and maybe kind of realizing this is the wrong direction?

有没有人在这方面做得更好，并且可能意识到这是错误的方向？

Edwin Chen: I would say I've always been very impressed by Anthropic.

Edwin Chen：我想说，Anthropic 一直给我留下了深刻的印象。

I think Anthropic takes a very principled view about what they do and don't care about and how they want their models to behave in a way that feels a lot more principle to me.

Interesting.: Are there any other big mistakes you think labs are making just that are kind of slowing things down or heading in the wrong direction?

有趣的是：您认为实验室是否还犯过其他重大错误，这些错误会减慢速度或走向错误的方向？

Where we've heard just chasing benchmarks, this engagement focus, is there anything else you're seeing of just like, "Okay, we got to work on this because it'll speed everything up"?

我们听说过只是追逐基准、这种参与焦点，但你是否还看到过其他类似的东西，比如“好吧，我们必须努力解决这个问题，因为它会加快一切速度”？

Edwin Chen: I think there is a question of what products they're building and whether those products themselves are something that kind of help or hurt humanity.

Edwin Chen：我认为存在一个问题，即他们正在制造什么产品，以及这些产品本身是否有助于或伤害人类。

I think a lot about Sora and...

我对索拉想了很多，还有……

Lenny Rachitsky: I was thinking that's what you're imagining.

Lenny Rachitsky：我以为这就是你的想象。

Edwin Chen: Yeah, what it entails, and so it's kind of interesting.

Edwin Chen：是的，它意味着什么，所以这很有趣。

It's like which companies would build Sora and which wouldn't?

就像哪些公司会建造 Sora，哪些公司不会？

Edwin Chen: And I think that answer to that...

Edwin Chen：我认为这个问题的答案......

Well, I don't know if answer is myself.

嗯，我不知道答案是不是我自己。

I have an idea in my head, but I think the answer to that question maybe reveals certain things about what kinds of AI models those companies want to build and what direction and what future they want to achieve, yeah, so I think about that a lot.

Lenny Rachitsky: The steel man argument there is, it's like fun, people want it, it'll help them generate revenue to grow this thing and build better models, it'll train data in an interesting way, it's also just really fun.

Edwin Chen: Yeah.

陈埃德温：是的。

I think it's almost like, do you care about how you get there?

And in the same way, so I made this tabloid analogy earlier, but would you sell tabloids in order to fund, I don't know, some other newspaper?

同样，我之前做了这个小报的类比，但是你会出售小报来资助其他报纸吗？我不知道？

Edwin Chen: Sure, like in some sense, if you don't care about the path, then you'll just do whatever it takes, but it's possible that it has negative consequences in of itself that will harm the long-term direction of what you're trying to achieve, and maybe it'll distract you from all the more important things, so yeah, I think that the path you take matters a lot as well.

Edwin Chen：当然，从某种意义上说，如果你不关心这条路，那么你就会不择手段，但它本身可能会产生负面后果，会损害你想要实现的目标的长期方向，也许它会分散你对所有更重要事情的注意力，所以是的，我认为你所走的路也很重要。

Lenny Rachitsky: Along these lines, you talked a bunch about this of just Silicon Valley and kind of the downsides of raising a lot of money being in the echo chamber.

Lenny Rachitsky：沿着这些思路，你谈到了很多关于硅谷的事情，以及在回声室中筹集大量资金的缺点。

What do you call it, the Silicon Valley machine?

你怎么称呼它，硅谷机器？

You talk about how it's hard to build important companies in this way and that you might actually be much more successful if you're not going down the VC path.

你谈到以这种方式建立重要的公司是多么困难，如果你不走风险投资的道路，你实际上可能会更成功。

Can you just talk about what you've seen in that experience and your advice essentially to founders, because they're always hearing?

您能否谈谈您在那次经历中看到的情况以及您对创始人的建议，因为他们总是在倾听？

Raise money from fancy VCs, move to Silicon Valley, what's kind of the countertake?

从高级风险投资公司筹集资金，搬到硅谷，什么样的反击？

Edwin Chen: Yes.

陈埃德温：是的。

So I've always really hated a lot of the Silicon Valley mantras.

所以我一直非常讨厌硅谷的很多口头禅。

The standard playbook is to get product market fit by pivoting every two weeks.

标准的策略是通过每两周调整一次来使产品适应市场。

And to chase growth and chase engagement with all of these dark patterns and to blitz scale by hiring as fast as possible.

为了追求增长并追求所有这些黑暗模式的参与，并通过尽快招聘来实现闪电式扩张。

And I've always disagreed.

而我一直不同意。

Edwin Chen: So yeah, I would say don't pivot.

Edwin Chen：是的，我想说不要转向。

Don't put scale.

不要放刻度。

Don't hire that Stanford grad who simply wants to add a hot company to your resume, just build the one thing only you could build, a thing that wouldn't exist without the insight and expertise that only you have.

不要雇用那个只想在你的简历中添加一家热门公司的斯坦福大学毕业生，只需构建只有你才能构建的东西，如果没有只有你拥有的洞察力和专业知识，它就不会存在。

Edwin Chen: And you see these buy to companies everywhere now.

Some founder who was doing crypto in 2020, and then pivoted to NFTs in 2022, and now they're an AI company.

一些创始人在 2020 年做加密货币，然后在 2022 年转向 NFT，现在他们是一家人工智能公司。

There's no consistency, there's no mission, they're just chasing valuations.

没有一致性，没有使命，他们只是追逐估值。

And I've always hated this because Silicon Valley loves to score on Wall Street for focusing on money.

But honestly, most of the Silicon Valley's chasing the same thing.

And so we stayed focused on our mission from day one, pushing that frontier of high quality complex data, and I've always loved that because I think startups...

因此，我们从第一天起就专注于我们的使命，推动高质量复杂数据的前沿，我一直很喜欢这一点，因为我认为初创公司......

Edwin Chen: I have this very romantic notion of startups.

Edwin Chen：我对创业公司有一个非常浪漫的想法。

Startups are supposed to be a way of taking big risks to build something that you really believe in.

初创公司应该是一种冒巨大风险来打造你真正相信的东西的方式。

But if you're constantly pivoting, you're not taking any risks.

但如果你不断地改变方向，你就不会冒任何风险。

You're just trying to make a quick buck.

你只是想赚快钱。

And if you fail because the market isn't ready yet, I actually think that's way better.

At least you took a swing at something deep, and novel, and hard instead of pivoting into another LLM wrapper company.

至少你尝试了一些有深度、新颖、困难的东西，而不是转向另一家法学硕士包装公司。

So yeah, I think the only way you build something that matters that's going to change the world is if you find a big idea you believe in and you say no to everything else.

所以，是的，我认为你建立一些能够改变世界的重要东西的唯一方法就是找到一个你相信的伟大想法并对其他一切说不。

Edwin Chen: So you don't keep on pivoting when it gets hard, you don't hire a team of 10 product managers because that's what every other cookie cutter startup does, you just keep building that one company that wouldn't exist without you.

Edwin Chen：所以，当事情变得困难时，你不会继续转型，你不会雇佣一个由 10 名产品经理组成的团队，因为这就是其他千篇一律的初创公司所做的，你只是继续建立一家没有你就不会存在的公司。

And I think there are a lot of people in Silicon Valley now who are sick of all the grift, who want to work on big things that matter with people who actually care, and I'm hoping that that would be the future of how we go with technology.

我认为硅谷现在有很多人厌倦了所有的欺骗，他们想要与真正关心的人一起做重要的事情，我希望这将是我们如何使用技术的未来。

Lenny Rachitsky: I'm actually working on a post right now with Terrence Rohan, this VC that I really like to work with, and we interviewed five people who picked really successful generational companies early and joined them as really early employees.

Lenny Rachitsky：实际上，我现在正在与 Terrence Rohan 一起工作，我非常喜欢与这位风险投资家合作，我们采访了五个人，他们很早就选择了真正成功的代际公司，并作为真正的早期员工加入了他们。

They joined OpenAI before anyone thought it was awesome, Stripe before anyone knew was awesome, and so we're looking for patterns of how people find these generational companies before anyone else, and it aligns exactly what you described, which is ambition.

他们在人们认为 OpenAI 很棒之前就加入了，在人们知道 Stripe 很棒之前就加入了，所以我们正在寻找人们如何先于其他人找到这些一代公司的模式，它与你所描述的完全一致，这就是雄心。

They have a wild ambition with what they want to achieve.

They're not, as you said, just kind of looking around for product market fit no matter what ends up being, and so I love that what you described very much aligns with what we're seeing there.

正如您所说，他们并不是只是四处寻找适合市场的产品，无论最终结果如何，所以我喜欢您所描述的内容与我们在那里看到的非常一致。

Edwin Chen: Yeah, I absolutely think that you have to have huge ambitions, and you have to have a huge belief in your idea that's going to change the world, and you have to be willing to double down and keep on doing whatever it takes to make it happen.

Edwin Chen：是的，我绝对认为你必须有远大的抱负，你必须对你的想法有巨大的信念，这将改变世界，你必须愿意加倍努力，继续不惜一切代价来实现它。

Lenny Rachitsky: I love how counter your narrative is to so many of the things people hear, and so I love that we're doing this.

I love that we're sharing this story.

我喜欢我们分享这个故事。

Lenny Rachitsky: Today's episode is brought to you by Coda.

Lenny Rachitsky：今天的节目由 Coda 为您带来。

I personally use Coda every single day to manage my podcast and also to manage my community.

我个人每天都使用 Coda 来管理我的播客和社区。

It's where I put the questions that I plan to ask every guest that's coming on the podcast, it's where I put my community resources, it's how I manage my workflows.

这是我提出问题的地方，我计划向播客中的每一位来宾提出问题，这是我放置社区资源的地方，这是我管理工作流程的方式。

Here's how Coda can help you.

以下是 Coda 如何帮助您。

Lenny Rachitsky: Imagine starting a project at work.

Lenny Rachitsky：想象一下在工作中启动一个项目。

And your vision is clear, you know exactly who's doing what, and where to find the data that you need to do your part.

In fact, you don't have to waste time searching for anything because everything your team needs from project trackers and OKRs, the documents and spreadsheets lives in one tab all in Coda.

事实上，您不必浪费时间搜索任何内容，因为您的团队所需的项目跟踪器和 OKR、文档和电子表格中的所有内容都位于 Coda 的一个选项卡中。

Lenny Rachitsky: With Coda's collaborative all in one workspace, you get the flexibility of docs, the structure of spreadsheets, the power of applications, and the intelligence of AI all in one easy to organize tab.

Lenny Rachitsky：借助 Coda 的协作式全合一工作区，您可以在一个易于组织的选项卡中获得文档的灵活性、电子表格的结构、应用程序的强大功能以及 AI 的智能。

Like I mentioned earlier, I use Coda every single day.

正如我之前提到的，我每天都使用 Coda。

And more than 50,000 teams trust Coda to keep them more aligned and focused.

If you're a startup team looking to increase alignment and agility, Coda can help you move from planning to execution in record time.

如果您是一个希望提高一致性和敏捷性的初创团队，Coda 可以帮助您在创纪录的时间内从计划转向执行。

Lenny Rachitsky: To try it for yourself, go to coda.io/lenny today and get six months free of the team planned for startups.

That's coda.io/lenny to get started for free and get six months of the team plan, coda.io/lenny.

Lenny Rachitsky: Slightly different direction, but something else that was maybe a counter narrative.

I imagine you watched the Dwarkesh and Richard Sutton podcast episode, and even if you didn't, they basically had this conversation, Richard Sutton.

我想你看过 Dwarkesh 和 Richard Sutton 的播客节目，即使你没有，他们基本上也进行了这样的对话，Richard Sutton。

He was a famous AI researcher, had this whole bitter lesson meme, and he talked about how LLMs almost are kind of a dead end, and he thinks we're going to really plateau around LLMs because of the way they learn.

他是一位著名的人工智能研究员，有整个痛苦的教训模因，他谈到法学硕士几乎是一个死胡同，他认为由于法学硕士的学习方式，我们将在法学硕士周围真正陷入停滞状态。

Lenny Rachitsky: What's your take there?

Lenny Rachitsky：你对此有何看法？

Do you think LLMs will get us to AGI or beyond, or do you think there's going to be something new or a big breakthrough that needs to get us there?

你认为法学硕士会让我们达到 AGI 或更高水平，还是你认为需要一些新的东西或重大突破来帮助我们实现这一目标？

Edwin Chen: I'm in the camp where I do believe that something new will be needed.

The way I think about it is when I think about training AI, I take a very...

I don't know if I would say biological point of view.

But I believe that in the same way that there's a million different ways that humans learn, we need to build models that can mimic all of those ways as well.

但我相信，就像人类有一百万种不同的学习方式一样，我们也需要建立可以模仿所有这些方式的模型。

And maybe they'll have a different distribution of the focuses that they have.

I know that it'll be different for humans, so maybe they have a different distribution, but we want to be able to mimic their learning abilities of humans and make sure that we have the algorithms and the data for models to learn in the same way.

我知道这对人类来说会有所不同，所以也许他们有不同的分布，但我们希望能够模仿人类的学习能力，并确保我们拥有模型以相同方式学习的算法和数据。

And so to the extent that LLMs have different ways of learning from humans, then yeah, I think something new will be needed.

Lenny Rachitsky: This connects to reinforcement learning.

This is something that you're big on and something I'm hearing more and more is just becoming a big deal in the world of post-training.

Can you just help people understand what is reinforcement learning and reinforcement learning environments, and why they're going to be more and more important in the future?

您能否帮助人们了解什么是强化学习和强化学习环境，以及为什么它们在未来会变得越来越重要？

Edwin Chen: Reinforcement learning is essentially training your model to reach a certain reward.

And let me explain what an RL environment is.

让我解释一下什么是 RL 环境。

An RL environment is essentially a simulation of real world.

强化学习环境本质上是对现实世界的模拟。

So think of it like building a video game with a fully fleshed out universe.

因此，可以把它想象成构建一个具有完全充实的宇宙的视频游戏。

Every character has a real story, every business has tools and data you can call, and you have all these different entities interacting with each other.

每个角色都有一个真实的故事，每个企业都有可以调用的工具和数据，并且所有这些不同的实体都可以相互交互。

Edwin Chen: So for example, we might build a world where you have a startup with Gmail messages, and Slack threads, and Jira tickets, and GitHub PRs, and a whole code base.

Edwin Chen：例如，我们可能会构建一个世界，其中您拥有一家拥有 Gmail 消息、Slack 线程、Jira 票证、GitHub PR 以及整个代码库的初创公司。

And then suddenly AWS goes down.

然后 AWS 突然宕机了。

And Slack goes down.

Slack 也衰落了。

And so, "Okay.

所以，“好吧。

Model, well, what do you do?" The model needs to figure it out.

模特，嗯，你是做什么的？”模型需要弄清楚这一点。

Edwin Chen: So we give them models tasks in these environments, we design interesting challenges for them, and then we run them to see how they perform.

Edwin Chen：所以我们给他们在这些环境中建立模型任务，为他们设计有趣的挑战，然后运行它们来看看它们的表现如何。

And then we teach them, we give them these rewards when they're doing a good job or a bad job.

然后我们教他们，当他们做得好或做得不好时，我们会给他们这些奖励。

Edwin Chen: And I think one of the interesting things is that these environments really showcase where models are weak at end-to-end tasks in real world.

Edwin Chen：我认为有趣的事情之一是这些环境确实展示了模型在现实世界中端到端任务中的弱点。

You have all these models that seem really smart on isolated benchmarks, they're good at single step tool calling.

所有这些模型在独立的基准测试中看起来都非常聪明，它们擅长单步工具调用。

They're good at single step instruction following.

他们擅长遵循单步指令。

But suddenly you dump them into these messy worlds where you have confusing Slack messages and tools they've never seen before, and they need to perform right actions and modify the and interact over longer time horizons where what they do in step one affects what they do in step 50.

但突然你把他们扔进了这些混乱的世界，在那里你有令人困惑的 Slack 消息和他们以前从未见过的工具，他们需要执行正确的操作并修改并在较长的时间范围内进行交互，他们在第一步中所做的事情会影响他们在步骤 50 中所做的事情。

And that's very different from these kind of academic single step environments that they've been in before, and so the model just fails catastrophically in all these crazy ways.

这与他们之前所处的学术单步环境非常不同，因此该模型在所有这些疯狂的方面都灾难性地失败了。

Edwin Chen: So I think these RL environments are going to be really interesting playgrounds for the models to learn from that will essentially be simulations and mimics in real world, and so they'll hopefully get better and better at real tasks compared to all these contrived environments.

Edwin Chen：所以我认为这些 RL 环境将成为模型学习的非常有趣的游乐场，本质上是现实世界中的模拟和模仿，因此与所有这些人为的环境相比，它们有望在实际任务中变得越来越好。

Lenny Rachitsky: So I'm trying to imagine what this looks like.

Lenny Rachitsky：所以我试着想象这是什么样子。

Essentially, it's like a virtual machine with, I don't know, a browser or a spreadsheet or something in it with like, I don't know, surge.com.

本质上，它就像一个虚拟机，里面有一个浏览器、一个电子表格或者类似我不知道的surge.com之类的东西。

Is that your website, surge.com?

Surge.com 那是你的网站吗？

Let's make sure we get that right.

让我们确保我们做对了。

Edwin Chen: So we are actually surgehq.ai.

Edwin Chen：所以我们实际上是surghq.ai。

Lenny Rachitsky: Surgehq.ai.

莱尼·拉奇茨基：Surgehq.ai。

Check it out.

一探究竟。

We're hiring it, I imagine.

我想我们正在雇用它。

Yes.

是的。

Okay.

好的。

So it's like, cool, here's surgehq.ai.

所以这就像，酷，这里是surghq.ai。

Your job, here's your job as an agent, let's say, is to make sure it stays up.

你的工作，比如说，你作为代理人的工作，就是确保它保持正常运转。

And then all of a sudden it goes down and the objective function is figure out why.

然后突然下降，目标函数就是找出原因。

Is that an example?

这是一个例子吗？

Edwin Chen: Yeah, so the objective function might be...

Edwin Chen：是的，所以目标函数可能是......

Or the goal of the task might be, okay, go figure out why and fix it.

或者任务的目标可能是，好吧，找出原因并解决它。

And so the objective function might be, it might be passing a series of unit tests, it might be writing a document like maybe it's a retro containing certain information that matches exactly what happened, there's all these different rewards that we might give it that determine whether or not it's succeeding, and so the models, we're basically teaching the models to achieve that reward.

所以目标函数可能是，它可能会通过一系列的单元测试，它可能会写一个文档，就像它可能是一个包含与所发生的情况完全匹配的某些信息的回顾，我们可能会给予它所有这些不同的奖励来确定它是否成功，所以模型，我们基本上是在教模型来实现该奖励。

Lenny Rachitsky: So essentially it's off and running.

Lenny Rachitsky：所以基本上它已经启动并运行了。

Here's your goal, figure out why the site went down and fix it.

这是您的目标，找出网站崩溃的原因并修复它。

And it just starts trying stuff, we're using everything, all the intelligence it's got, it makes mistakes, you kind of help it along the way, reward it if it's doing the right sort of thing.

它只是开始尝试一些东西，我们正在使用一切，它所拥有的所有智能，它会犯错误，你一路上帮助它，如果它做了正确的事情，就奖励它。

And so what you're describing here is this is the next phase of models becoming smarter.

因此，您在这里所描述的是，这是模型变得更加智能的下一阶段。

More RL environments focused on very specific tasks that are economically valuable, I imagine.

我想更多的强化学习环境专注于具有经济价值的非常具体的任务。

Edwin Chen: Yeah, so just in the same way that there were all these different methods for models of learning in the past, originally we had SFT and RHF, and then we had rubrics and verifiers.

Edwin Chen：是的，就像过去有各种不同的学习模型方法一样，最初我们有 SFT 和 RHF，然后我们有评估标准和验证器。

This is the next stage, and it's not the case that the previous methods are obsolete, this is, again, just a different form of learning that compliments all the previous types, so it's just like a different skilled model not only to learn how to do.

这是下一个阶段，并不是说以前的方法已经过时，这又是一种不同的学习形式，它补充了所有以前的类型，所以它就像一个不同的技能模型，不仅仅是学习如何做。

Lenny Rachitsky: And so in this case, it's less some physics PhD sitting around talking to a model, correcting it, giving it evals of here's what the correct answer is, creating rubrics and things like that.

Lenny Rachitsky：所以在这种情况下，并不是某个物理学博士坐在一起与模型交谈，纠正它，评估正确答案是什么，创建标题之类的东西。

More it's like this person now designing an environment.

更像是这个人现在正在设计一个环境。

So another example I've heard is like a financial analyst.

我听过的另一个例子是金融分析师。

Just like, "Here's an Excel spreadsheet, here's your goal, figure out our profit and loss," or whatever.

就像，“这是一个 Excel 电子表格，这是您的目标，计算出我们的损益”，或者其他什么。

And so this expert now, instead of just sitting around writing rubrics, they're designing this RL environment.

因此，这位专家现在不再只是坐下来编写标题，而是正在设计这个 RL 环境。

Edwin Chen: Yeah, exactly.

埃德温·陈：是的，完全正确。

So that financial analyst might create a spreadsheet, they may create certain tools that the model needs to call in order to help fill out that spreadsheet, like it might be, okay, the model needs to access Bloomberg terminal.

因此，财务分析师可能会创建一个电子表格，他们可能会创建模型需要调用的某些工具，以帮助填写该电子表格，就像模型需要访问彭博终端一样。

It needs to learn how to use it.

它需要学习如何使用它。

And it needs to learn how to use this calculator.

它需要学习如何使用这个计算器。

And it needs to learn how to pour on this calculation.

它需要学习如何进行这种计算。

So it has all these tools that it has access to.

所以它拥有它可以访问的所有这些工具。

Edwin Chen: And then the reward might be...

Edwin Chen：然后奖励可能是......

Okay, it's like maybe I will download that spreadsheet and I want to see, does cell B22 contain the correct profit and loss number?

好吧，也许我会下载该电子表格，然后我想看看单元格 B22 是否包含正确的损益数字？

Or does tab number two contain this piece of information?

或者第二个选项卡包含这条信息吗？

Lenny Rachitsky: And what's interesting, this is a lot closer to how humans learn.

Lenny Rachitsky：有趣的是，这更接近人类的学习方式。

We just try stuff, figure out what's working and what's not.

我们只是尝试一些东西，找出什么有效，什么无效。

You talk about how trajectories are really important to this.

您谈到轨迹对此非常重要。

It's not just here's the goal and here's the end, it's like every step along the way.

这不仅仅是目标和终点，更是一路上的每一步。

Can you just talk about what trajectories are and why that's important to this?

您能谈谈什么是轨迹以及为什么它对此很重要吗？

Edwin Chen: I think one of the things that people don't realize is that sometimes even though the model reaches the correct answer, it does so in all these crazy ways.

Edwin Chen：我认为人们没有意识到的一件事是，有时即使模型得出了正确的答案，它也会以所有这些疯狂的方式做到这一点。

So it may have in the intermediate trajectory, it may have tried 50 different times and failed, but eventually it just kind of randomly lands on a correct number.

因此，它可能在中间轨迹中，可能尝试了 50 次不同的次数并失败了，但最终它只是随机落在正确的数字上。

Or maybe it is...

或者也许是...

Edwin Chen: Sometimes it just does things very inefficiently or it almost reward-hacks a way to get at the correct answer, and so I think paying attention to the directory is actually really important.

Edwin Chen：有时它只是做事效率非常低，或者几乎是奖励黑客获得正确答案的方法，所以我认为关注目录实际上非常重要。

And I think it's also really important because some of these trajectories can be very long.

我认为这也非常重要，因为其中一些轨迹可能会很长。

And so if all you're doing is checking whether or not the model reaches the final answer, it's like there's all this information about how the model behaved in the immediate step that's missing.

因此，如果您所做的只是检查模型是否达到最终答案，那么就好像缺少有关模型在立即步骤中的行为方式的所有信息。

Edwin Chen: Sometimes you want models to get to the correct answer by reflecting on what it did.

Edwin Chen：有时你希望模型通过反思它所做的事情来得到正确的答案。

Sometimes you want it to get it at the correct answer by just one-shotting it.

有时您希望它只需一击即可获得正确答案。

And if you ignore all of that, it's just like teaching it...

如果你忽略所有这些，那就就像教它一样......

just missing a lot of the information that you could be teaching a model to do.

只是缺少了很多可以教模型做的信息。

Lenny Rachitsky: I love that.

莱尼·拉奇茨基：我喜欢这个。

Yeah, it tries a bunch of stuff and eventually gets it right.

是的，它尝试了很多东西，最终得到了正确的结果。

You don't want it to learn this is the way to get there.

你不希望它知道这是到达那里的方法。

There's often a much more efficient way of doing it.

通常有一种更有效的方法来做到这一点。

Lenny Rachitsky: You mentioned all the kind of the steps we've taken along the journey of helping models get smarter.

Lenny Rachitsky：您提到了我们在帮助模型变得更智能的过程中所采取的所有步骤。

Since you've been so close to this for so long, I think this is going to be really helpful for people.

由于您长期以来一直关注这一点，我认为这对人们非常有帮助。

What's kind of like been the steps along the way from the first post-training that has most helped models advance?

从第一次训练后最能帮助模型进步的步骤是怎样的？

Where do evals fit in the RL environments?

评估适合 RL 环境中的什么位置？

Just like what's been the steps and now we're heading towards RL environments?

就像我们现在正在迈向 RL 环境的步骤一样？

Edwin Chen: Originally, the way models started getting post-trained was purely through SFT.

Edwin Chen：最初，模型开始进行后训练的方式纯粹是通过 SFT。

And-

还有——

Edwin Chen: So SFT stands for supervised fine-tuning.

Edwin Chen：所以SFT代表监督微调。

So again, I think often in terms of these human analogies, and so SFT is a lot like mimicking a master and copying what they do.

再说一次，我经常用这些人类类比来思考，所以 SFT 很像模仿大师并复制他们所做的事情。

Edwin Chen: And then RLHF became very dominant.

Edwin Chen：然后 RLHF 变得非常有统治力。

And analogy there would be like sometimes you learn by writing 55 different essays and someone telling you which one they liked the most.

打个比方，就像有时你通过写 55 篇不同的文章，然后有人告诉你他们最喜欢哪一篇来学习。

Edwin Chen: And then I think over the past year or so, rubrics and verifiers have become very important.

Edwin Chen：然后我认为在过去一年左右的时间里，标题和验证者变得非常重要。

And rubrics and verifiers are like learning by being graded and getting detailed feedback on where you went wrong.

评分标准和验证者就像通过评分来学习并获得关于哪里出错的详细反馈。

Lenny Rachitsky: And those are evals, another word for that?

Lenny Rachitsky：这些是评估，即评估的另一种说法？

Edwin Chen: Yeah.

陈埃德温：是的。

So I think evals often covers two terms.

所以我认为评估通常涵盖两个术语。

One is you are using the evaluations for training because you're evaluating whether or not the model did a good job, and when it does do a good job, you're rewarding it.

一是您使用评估进行训练，因为您正在评估模型是否做得很好，当它确实做得很好时，您就会奖励它。

Edwin Chen: And then there's this other notion of evals where you're trying to measure the model's progress like, okay, yeah, I have five different candidate checkpoints and I want to pick the one that's best in order to release it to the public.

Edwin Chen：然后还有另一种评估概念，你试图衡量模型的进度，就像，好吧，是的，我有五个不同的候选检查点，我想选择最好的一个，以便将其发布给公众。

So going to run all these evals on these five different checkpoints in order to decide which one is best.

因此，将对这五个不同的检查点运行所有这些评估，以确定哪一个最好。

Edwin Chen: Yeah, and now we have RL environments, so this is kind of like a hot new thing.

Edwin Chen：是的，现在我们有了强化学习环境，所以这有点像一个热门的新事物。

Lenny Rachitsky: Awesome.

莱尼·拉奇茨基：太棒了。

So what I love about this business journey is just there's always something new.

所以我喜欢这次商业之旅的原因就是总是有新的东西。

There's always this like, okay.

总有这样的事，好吧。

We're getting so good at just all this beautiful data for companies and now they need something completely different.

我们已经非常擅长为公司提供所有这些美丽的数据，而现在他们需要完全不同的东西。

Now we're setting up all these virtual machines for them and all these different use cases.

现在我们正在为它们和所有这些不同的用例设置所有这些虚拟机。

Lenny Rachitsky: And it feels like that's a big part of this industry you're in, it's just adapting to what labs are asking for.

Lenny Rachitsky：感觉这是您所在行业的重要组成部分，它只是适应实验室的要求。

Edwin Chen: Yeah.

陈埃德温：是的。

So I really do think that we are going to need to build a suite of products that reflect a million different ways that humans learn.

所以我确实认为我们需要构建一套产品来反映人类一百万种不同的学习方式。

Edwin Chen: Like for example, think about becoming a great writer.

Edwin Chen：比如说，考虑成为一名伟大的作家。

You don't become great by memorizing a bunch of grammar rules.

记住一堆语法规则并不能让你变得伟大。

You become great by reading great books, and you practice writing, and you get feedback from your teachers and from the people who buy your books in a bookstore and leave reviews.

你通过阅读好书而变得伟大，你练习写作，你从你的老师和在书店购买你的书并留下评论的人那里得到反馈。

And you notice what works and what doesn't.

你会注意到什么有效，什么无效。

And you develop taste by being exposed to all of these masterpieces and also just terrible writing.

通过接触所有这些杰作和糟糕的写作，你可以培养品味。

So you learn through this endless cycle of practicing reflection, and each type of learning that you have, again, these are all very different methods of learning to become a great writer, so just in the same way that...

因此，你通过这种无休止的练习反思循环来学习，以及你所拥有的每种类型的学习，同样，这些都是学习成为一名伟大作家的非常不同的方法，所以就像……

it's a thousand different ways that the great writer becomes great, I think there's going to be a thousand different ways that AI need to learn.

伟大的作家有一千种不同的方式变得伟大，我认为人工智能需要学习一千种不同的方式。

Lenny Rachitsky: It's so interesting this just ends up being just like humans in so many ways.

Lenny Rachitsky：这太有趣了，它最终在很多方面都像人类一样。

It makes sense because in a sense, neural networks, deep learning is modeled after how humans have learned and how our brains operate, but it's interesting just to make them smarter.

这是有道理的，因为从某种意义上说，神经网络、深度学习是按照人类的学习方式和大脑的运作方式建模的，但仅仅让它们变得更聪明就很有趣。

It's how do we come closer to how humans learn more and more?

我们如何才能越来越接近人类学习的方式？

Edwin Chen: Yeah, it's almost like maybe the end goal is just throwing you into the environment and just seeing how you evolve.

Edwin Chen：是的，最终目标可能就是把你扔到这个环境中，看看你如何进化。

But within that evolution, there's all these different sub-learning mechanisms.

但在这种演变中，存在着所有这些不同的子学习机制。

Lenny Rachitsky: Yeah, which is kind of what we're doing now, so that's really interesting.

Lenny Rachitsky：是的，这就是我们现在正在做的事情，所以这真的很有趣。

This might be the last step until we hit AGI.

这可能是我们实现 AGI 之前的最后一步。

Along these lines, something that's really unique to Surge that I learned is you guys have your own research team, which I think is pretty rare, talk about just why that's something you guys have invested in and what has come out of that investment.

Edwin Chen: Yeah, so I think that stems from my own background.

Edwin Chen：是的，所以我认为这源于我自己的背景。

My own background is as a researcher.

我自己的背景是一名研究员。

And so I've always cared fundamentally about pushing the industry and pushing the research community and not just about revenue.

因此，我一直从根本上关心推动行业和研究界的发展，而不仅仅是收入。

And so I think what our research team does is a couple different things.

所以我认为我们的研究团队所做的是几件不同的事情。

Edwin Chen: So we almost have two types of researchers at our company.

Edwin Chen：所以我们公司几乎有两种类型的研究人员。

One is our forward-deployed researchers who are often working hand in hand with our customers to help them understand their models.

其中之一是我们前沿部署的研究人员，他们经常与我们的客户携手合作，帮助他们理解他们的模型。

So we will work very closely with the customers to help them understand, "Okay, this is where your model is today.

因此，我们将与客户密切合作，帮助他们理解，“好吧，这就是您今天的模型。

This is where you're lagging behind all the competitors, these are some ways that you could be improving in the future, given your goals, and we're going to design these data sets, these evaluation methods, these training techniques to make your models better." So this very collaborative notion of working with our customers being researched by themselves, just a little bit more focused on the data side, and working hand on hand with them to do whatever it takes to make them the best.

这就是你落后于所有竞争对手的地方，考虑到你的目标，这些是你未来可以改进的一些方法，我们将设计这些数据集、这些评估方法、这些训练技术来让你的模型变得更好。”因此，这种非常协作的理念是与我们的客户合作，他们自己进行研究，稍微关注数据方面，并与他们携手合作，尽一切努力使他们做到最好。

Edwin Chen: And then we also have our internal researchers.

Edwin Chen：我们还有内部研究人员。

So our internal researchers are focused on slightly different things.

因此，我们的内部研究人员关注的事情略有不同。

So they are focused on building better benchmarks and better leaderboards.

因此，他们专注于建立更好的基准和更好的排行榜。

Edwin Chen: So I've talked a lot about how I worry that the leaderboards and benchmarks out there today are steering models in the wrong direction, so yeah, so the question is, how do we fix that?

And so that's what our research team is focused focused really heavily on right now.

这就是我们的研究团队目前重点关注的问题。

So they're working a lot on that.

所以他们在这方面做了很多工作。

Edwin Chen: And they're also working on these other things like, "Okay, we need to train our own models to see what types of data performs the best, what types of people perform the best." And so they're also working on all these training techniques and evaluation of our own data sets to improve our data operations and the internal data products that we have that determine what makes something good quality.

Edwin Chen：他们还在研究其他事情，比如“好吧，我们需要训练我们自己的模型，看看什么类型的数据表现最好，什么类型的人表现最好。” And so they're also working on all these training techniques and evaluation of our own data sets to improve our data operations and the internal data products that we have that determine what makes something good quality.

Lenny Rachitsky: It's such a cool thing because I don't think basically the labs have researchers helping them advance AI.

Lenny Rachitsky：这是一件很酷的事情，因为我认为实验室基本上没有研究人员帮助他们推进人工智能。

I imagine it's pretty rare for a company like yours to have researchers actually doing primary research on AI.

我想像你们这样的公司很少有研究人员真正对人工智能进行初步研究。

Edwin Chen: Yeah, I think it's just because it's something I've fundamentally always cared about.

Edwin Chen：是的，我想这只是因为这是我从根本上一直关心的事情。

I often think about us more like a research lab than a startup because that is my goal.

我经常认为我们更像是一个研究实验室，而不是一家初创公司，因为这是我的目标。

It's kind of funny, but I've always said I would rather be Terrence Tau than Warren Buffett, so that notion of creating research that pushes the frontier forward and not just getting some valuation, that's always been what drives me.

Lenny Rachitsky: And it's worked out.

Lenny Rachitsky：事情已经解决了。

That's the beautiful thing about this.

这就是这件事的美妙之处。

You mentioned that you were hiring researchers, is there anything there you want to share folks you're looking for?

您提到您正在招聘研究人员，您有什么想与您分享的吗？

Edwin Chen: So we look for people who are just fundamentally interested in dataset all day.

Edwin Chen：所以我们寻找整天对数据集感兴趣的人。

So types of people who could literally spend 10 hours digging through a dataset, and playing around with models, and thinking, "Okay, yeah, this is where I think the model's failing," this is the kind of a behavior you want the model to have instead, and just this aspect of being very hands-on and thinking about the qualitative aspects of models and not just the quantitative parts.

So again, it's like this aspect of being hands-on with data and not just caring about these kind of abstract algorithms.

再说一遍，这就像亲自动手处理数据，而不仅仅是关心这些抽象算法。

Awesome.: I want to ask a couple broad AI kind of market questions.

太棒了：我想问一些广泛的人工智能市场问题。

What else do you think is coming in the next couple of years that people are maybe not thinking enough about or not expecting in terms of where AI is heading?

What's going to matter?

有什么关系呢？

Edwin Chen: I think one of the things that's going to happen in the next few years is that the models are actually going to become increasingly differentiated because of the personalities and behaviors that the different labs have and the kind of objective functions that they are optimizing their models for.

I think it's one thing I didn't appreciate a year or so ago.

我认为这是大约一年前我不欣赏的一件事。

Edwin Chen: A year or so ago, I thought that all of the AI models would essentially become very commoditized.

Edwin Chen：大约一年前，我认为所有的人工智能模型基本上都会变得非常商品化。

They would all behave like each other, and sure, one of them might be slightly more intelligent in one way today, but sure, the other ones would catch up in the next few months.

But I think over the past year, I've realized that the values that the companies have will shape the model.

但我认为在过去的一年里，我意识到公司的价值观将塑造这种模式。

Edwin Chen: So let me give you an example.

Edwin Chen：让我举个例子。

So I was asking Claude to help me draft an email the other day, and it went through 30 different versions.

所以那天我请 Claude 帮我起草一封电子邮件，它经历了 30 个不同的版本。

And after 30 minutes, yeah, I think it really crafted me the perfect email, and I sent it.

30 分钟后，是的，我认为它确实为我制作了一封完美的电子邮件，然后我发送了它。

But then I realized that I spent 30 minutes doing something that didn't matter at all.

但后来我意识到我花了30分钟做了一些根本不重要的事情。

Sure, now I got the perfect email, but I spent 30 minutes doing something I wouldn't have worried at all before, and this email probably didn't even move the needle on anything anyways.

Edwin Chen: So I think there's a deep question here, which is, if you could choose the perfect model behavior, which model would you want?

Edwin Chen：所以我认为这里有一个深刻的问题，那就是，如果你可以选择完美的模型行为，你会想要哪个模型？

Do you want a model that says, "You're absolutely right.

你想要一个模型说：“你完全正确。

There are definitely 20 more ways to improve this email," and it continues for 50 more iterations.

肯定还有 20 种以上的方法来改进这封电子邮件”，并且它还会继续进行 50 次以上的迭代。

And it sucks up all your time and engagement.

它会占用你所有的时间和参与度。

Or do you want a model that's optimizing for your time and productivity and just says, "No, you need to stop.

或者你想要一个能够优化你的时间和生产力的模型，然后只是说：“不，你需要停下来。

Your email's great.

你的电子邮件很棒。

Just send it and move on with your day"?

只需发送即可继续您的一天”？

Edwin Chen: And again, just because...

Edwin Chen：再说一次，只是因为......

In the same way that there's like a kind of a fork in a road between how you could choose how your model behaves for this question, it's like for every other question that models have, the kind of behavior that you want will fundamentally affect it.

Edwin Chen: It's almost like in the same way that when Google builds a search engine, it's very different from how Facebook would build a search engine, which is very different from how Apple would build a search engine.

They all have their own principles and values and things that they're trying to achieve in the world that shape all the products that they're going to build.

And in the same way, I think all the will start behaving very differently too.

同样，我认为所有人的行为也会开始变得非常不同。

Lenny Rachitsky: That is incredibly interesting.

Lenny Rachitsky：这非常有趣。

You already see that with Grok.

您已经在 Grok 中看到了这一点。

It's got a very different personality and a very different approach to answering questions.

它有非常不同的个性和非常不同的回答问题的方法。

And so what I'm hearing is you're going to see more of this differentiation.

所以我听到的是，你会看到更多这种差异。

Lenny Rachitsky: Kind of another question along these lines, what do you think is most under-hyped in AI that you think maybe people aren't talking enough about that is really cool?

And what do you think is over-hyped?

您认为什么被过度炒作了？

Edwin Chen: So I think one of the things that's under-hyped is the built-in products that all of the chatbots are going to start having.

Edwin Chen：所以我认为没有被充分宣传的事情之一是所有聊天机器人都将开始拥有的内置产品。

I've always been a huge fan of Claude's artifacts.

我一直是克劳德手工艺品的忠实粉丝。

And I think it just works really well.

我认为它效果非常好。

And actually the other day, I don't know if it's a new feature or not, but it asked me to help me create an email, and then it just created...

实际上有一天，我不知道这是否是一个新功能，但它要求我帮助我创建一封电子邮件，然后它就创建了......

So it didn't quite work because it didn't allow me to send the email.

所以它不太有效，因为它不允许我发送电子邮件。

But what it created instead was like a little, I don't know what we call it, like a little box where I could click on it and it would just text someone that did this message.

And I think that concept of taking artifacts to the next level where you just have these mini apps, mini UIs within the chatbots themselves, I feel like people aren't talking enough about that.

So I think that that's one under-hyped area.

所以我认为这是一个未被充分宣传的领域。

Edwin Chen: And in terms of over-hyped areas, I definitely think that vibe coding is over-hyped.

Edwin Chen：就过度炒作的领域而言，我绝对认为氛围编码被过度炒作了。

I think people don't realize how much it's going to make your systems unmaintainable in the long-term and they simply dump this code into their code bases if this seems to work out right now, so I kind of worry about the future of coding.

It's just going to keep on happening.

它只会继续发生。

Lenny Rachitsky: These are amazing answers.

Lenny Rachitsky：这些都是令人惊奇的答案。

On that first point, there's something I actually asked.

关于第一点，我实际上问过一些事情。

I have the chief product officer of Anthropic and OpenAI, Kevin Weil and Mike Krieger on the podcast, and I asked them just like, "As a product team, you have this gigabrain intelligence.

Anthropic 和 OpenAI 的首席产品官 Kevin Weil 和 Mike Krieger 在播客上，我问他们，“作为一个产品团队，你们拥有这种千兆脑智能。

How long do you even need product teams?" You think this AI will just create the product for you.

你还需要产品团队多久？”你认为这个人工智能只会为你创造产品。

"Here's what I want." It's like the next level of vibe coding.

“这就是我想要的。”这就像下一个级别的氛围编码。

It's just like tell it, "Here's what I want," and it's just building the product and involving the product as you're using it.

这就像告诉它，“这就是我想要的”，它只是构建产品并在您使用它时涉及该产品。

And it feels like that's what you're describing is where we might be heading.

感觉你所描述的就是我们可能前进的方向。

Edwin Chen: Yeah, I think there's a very powerful notion where it helps people just achieve their ideas in a much cooler way.

Edwin Chen：是的，我认为有一个非常强大的概念，它可以帮助人们以更酷的方式实现他们的想法。

Lenny Rachitsky: Something we haven't gotten into that I think is really interesting is just the story of how you got to starting Surge.

Lenny Rachitsky：我们还没有讨论过一些我认为真正有趣的事情，那就是你如何创办 Surge 的故事。

You have a really unique background.

你有一个非常独特的背景。

I always think about these...

我时常思考这些...

Brian Armstrong, the founder of Coinbase, once gave this talk that has really stuck with me where he kind of talked about how his very unique background allowed him to start Coinbase.

Coinbase 的创始人 Brian Armstrong 曾经做过一次让我印象深刻的演讲，他谈到了他独特的背景如何让他创办了 Coinbase。

He had a economics background, he had a cryptography experience, and then he was an engineer.

他有经济学背景，有密码学经验，然后他是一名工程师。

And it's like the perfect Venn diagram for starting Coinbase, and I feel like you have a very similar story with Surge.

这就像启动 Coinbase 的完美维恩图，我觉得 Surge 的故事非常相似。

Talk about that, your background there, and how that led to Surge.

谈谈这个，你在那里的背景，以及这如何导致了 Surge。

Edwin Chen: Going way back, I was always fascinated by math and language when I was a kid.

Edwin Chen：回想起来，当我还是个孩子的时候，我总是对数学和语言着迷。

I went to MIT because it's obviously one of the best places for math and CS, but also because it's the home of Noam Chomsky.

我去麻省理工学院是因为它显然是数学和计算机科学最好的地方之一，而且还因为它是诺姆·乔姆斯基的故乡。

My dream in school was actually to find some underlying theory connecting all these different fields.

我在学校的梦想实际上是找到一些连接所有这些不同领域的基本理论。

Edwin Chen: And then I became a researcher at Google, and Facebook, and Twitter, and I just kept running into the same problem over and over again.

Edwin Chen：然后我成为了谷歌、Facebook 和 Twitter 的研究员，我只是一次又一次地遇到同样的问题。

It was impossible to get the data that we needed to train our models.

我们不可能获得训练模型所需的数据。

So I was always this huge believer in the need for high quality data, and then GPT-3 came out in 2020.

因此，我始终坚信需要高质量数据，然后 GPT-3 于 2020 年问世。

And I realized that, yeah, if we wanted to take things to the next level and build models that could code, and use tools, and tell jokes, and write poetry, and solve , and cure cancer, then yeah, we were going to need a completely new solution.

我意识到，是的，如果我们想将事情提升到一个新的水平并构建可以编码、使用工具、讲笑话、写诗、解决和治愈癌症的模型，那么是的，我们将需要一个全新的解决方案。

Edwin Chen: The thing that always drove me crazy when I was at all these companies was we had a full power of the human mind in front of us, and all the data students out there were focused on really simple things like image labeling.

Edwin Chen：当我在所有这些公司工作时，总是让我发疯的事情是，我们面前拥有人类思维的全部力量，并且所有数据学生都专注于非常简单的事情，例如图像标签。

So I wanted to build something focus on all these advanced, complex use cases instead that would really help us build our next generation models.

因此，我想构建一些专注于所有这些高级、复杂用例的东西，而不是真正帮助我们构建下一代模型。

So yeah, I think my background in kind of across math, and computer science, and linguistics really informed what I always wanted to do, and so I started Surge a month later with our one mission to basically build the use cases that I thought were going to be needed to push the frontier of AI.

所以，是的，我认为我在数学、计算机科学和语言学方面的背景确实告诉了我一直想做的事情，所以一个月后我开始了 Surge，我们的一个使命是基本上构建我认为推动人工智能前沿所需的用例。

Lenny Rachitsky: And you said a month later, a month later after what?

Lenny Rachitsky：你说一个月后，一个月后什么之后？

Oh, okay.

哦好的。

Wow.

哇。

Okay.

好的。

Yeah.

是的。

A great decision.: What just kind of drives you at this point of...

一个伟大的决定：此时此刻是什么驱使着你......

Other than just the epic success you're having, what keeps you motivated to keep building this and building something in this space?

除了您所取得的巨大成功之外，是什么让您有动力继续构建这个领域并在这个领域构建一些东西？

Edwin Chen: I think I'm a scientist at heart.

Edwin Chen：我认为我本质上是一名科学家。

I always thought I was going to become this math or CS professor and work on trying to understand the universe, and language, and the nature of communication.

我一直以为我会成为数学或计算机科学教授，努力理解宇宙、语言和交流的本质。

It's kind of funny, but I always had this fanciful dream where if aliens ever came to visit Earth and we need to figure out how to communicate with them, I wanted to be the one the government would call.

这有点有趣，但我一直有一个幻想，如果外星人来到地球，我们需要弄清楚如何与他们沟通，我想成为政府打电话的那个人。

And I'd use all this fancy math, and computer science, and linguistics to decipher it.

我会使用所有这些奇特的数学、计算机科学和语言学来破译它。

Edwin Chen: So even today, what I love doing most is every time a new model is released, we'll actually do a really deep dive into the model itself.

Edwin Chen：所以即使在今天，我最喜欢做的就是每次发布新模型时，我们实际上都会深入研究模型本身。

I'll play around with it, I'll run evals, I'll compare where it's improved, where it's arrest, I'll create this really deep dive analysis that we send our customers.

我会尝试它，我会运行评估，我会比较它的改进之处，它的缺陷，我会创建我们发送给客户的真正深入的深入分析。

And it's actually kind of funny because a lot of times we'll say it's from a data science team, but often it's actually just from me.

这实际上有点有趣，因为很多时候我们会说它来自数据科学团队，但实际上通常只是我自己。

Edwin Chen: And I think I could do this all day.

Edwin Chen：我想我可以做一整天。

I have a very hard time being in meetings all day.

我很难整天参加会议。

I'm terrible at sales, I'm terrible at doing the typical CEO things that people expect you to do, but I love writing these analyses.

我不擅长销售，也不擅长做人们期望你做的典型首席执行官的事情，但我喜欢写这些分析。

I love jamming with our research team about what we're seeing, sometimes I'll be up until 3:00 AM just talking on the phone with somebody on the research team and model.

我喜欢与我们的研究团队讨论我们所看到的内容，有时我会一直工作到凌晨 3:00，只是与研究团队和模型中的人员通过电话交谈。

So I love that I still get to be really hands-on, working on the data and the science all day.

所以我喜欢我仍然能够真正亲自动手，整天研究数据和科学。

And I think what drives me is that I want Surge to play this critical role in the future of AI, which I think is also the future of humanity.

我认为驱使我的是我希望 Surge 在人工智能的未来中发挥关键作用，我认为这也是人类的未来。

We have these really unique perspectives on data, and language, and quality, and how to measure all of this, and how to ensure it's all going on the right path.

我们对数据、语言和质量，以及如何衡量所有这些，以及如何确保一切都在正确的道路上进行，有着非常独特的视角。

And I think we're uniquely unconstrained by all of these influences that can sometimes steer companies in a negative direction.

我认为我们独特地不受所有这些有时会引导公司走向负面方向的影响的限制。

Edwin Chen: Like what I was saying earlier, we built Surge a lot more like a research lab than a typical startup.

Edwin Chen：就像我之前所说的，我们建立的 Surge 更像是一个研究实验室，而不是一家典型的初创公司。

So we care about curiosity and long-term incentives and intellectual rigor, and we don't care as much about quarterly metrics and what's going to look good in a .

因此，我们关心的是好奇心、长期激励和智力严谨性，而我们不太关心季度指标以及 .

And so my goal is to take all these unique things about us as a company and use that to make sure that we're shaping AI in a way that's really beneficial for our species in the long term.

因此，我的目标是利用我们作为一家公司的所有这些独特之处，并利用它们来确保我们以一种从长远来看对我们人类真正有益的方式塑造人工智能。

Lenny Rachitsky: What I'm realizing in this conversation is just how much influence you have and companies like yours have on where AI heads.

Lenny Rachitsky：在这次谈话中我意识到你和像你这样的公司对人工智能的发展方向有多大的影响力。

The fact that you help labs understand where they have gaps and where they need to improve, and it's not just everyone looks at just like the heads of OpenAI and Anthropic and all these companies as they're the ones ushering in AI, but what I'm hearing here is you have a lot of influence on where things head too.

事实上，你帮助实验室了解他们的差距和需要改进的地方，不仅仅是每个人都像 OpenAI 和 Anthropic 以及所有这些公司的负责人一样看待他们，因为他们是人工智能的引领者，但我在这里听到的是，你对事情的发展也有很大的影响力。

Edwin Chen: Yeah, I think there's this really powerful ecosystem where, honestly, people just don't know where models are headed and how they want to shape them yet and how they want humanity kind of like play a role in the future of all of this, and so I think there's a lot of opportunity to just continue shaping the discussion.

Edwin Chen：是的，我认为有一个非常强大的生态系统，老实说，人们只是不知道模型的发展方向，以及他们想要如何塑造它们，以及他们希望人类如何在所有这一切的未来中发挥作用，所以我认为有很多机会继续塑造讨论。

Lenny Rachitsky: Along that thread, I know you have a very strong thesis on just why this work matters to humanity and why this is so important, talk about that.

Lenny Rachitsky：沿着这条线索，我知道你有一个非常有力的论点，说明为什么这项工作对人类很重要，为什么它如此重要，请谈谈。

Edwin Chen: I'll get a bit philosophical here, but I think the question itself is a bit philosophical, so bear with me.

Edwin Chen：我会在这里讲一点哲学，但我认为这个问题本身就有点哲学，所以请耐心听我说。

So the most straightforward way of thinking about what we do is we train and evaluate AI.

因此，思考我们所做的事情最直接的方法是训练和评估人工智能。

But there's a deeper mission that I often think about, which is helping our customers think about their dream objective functions.

但我经常思考一个更深层次的使命，那就是帮助我们的客户思考他们梦想的目标函数。

Like yeah, what kind of model do they want their model to be?

是的，他们希望自己的模特是什么样的？

And once we help them do that, we'll help them train their model to reach their north star and we'll help them measure that progress.

一旦我们帮助他们做到这一点，我们将帮助他们训练他们的模型以达到他们的北极星，我们将帮助他们衡量这一进展。

But it's really hard because objective functions are really rich and complex.

但这确实很难，因为目标函数非常丰富且复杂。

It's kind of like the difference between having a kid and asking them, "Okay, what test do you want to pass?

这有点像生孩子和问他们“好吧，你想通过什么测试？

Do you want them to get a high score on SAT and write a really good college essay?" That's a simplistic version versus what kind of person do you want them to grow up to be?

你想让他们在 SAT 考试中取得高分并写出一篇非常好的大学论文吗？”这是一个简单化的版本，而不是你希望他们长大成为什么样的人？

Will you be happy if they're happy no matter what they do or are you hoping they'll go to a good school and be financially successful?

如果他们无论做什么都快乐，或者你希望他们能上一所好学校并在经济上取得成功，你会感到高兴吗？

Edwin Chen: And again, if you take that notion, it's like, okay, how do you define happiness?

Edwin Chen：再说一次，如果你接受这个概念，那就像是，好吧，你如何定义幸福？

How do you measure whether they're happy?

你如何衡量他们是否快乐？

How do you measure whether they're financially successful?

您如何衡量他们在财务上是否成功？

It's a lot harder than something measuring whether or not you're getting a high score on the SAT, and what we're doing is we want to help our customers reach, again, their dream north stars and figure out how to measure them.

这比衡量你是否在 SAT 考试中获得高分要困难得多，而我们正在做的是，我们希望帮助我们的客户再次实现他们梦想的北极星，并弄清楚如何衡量它们。

And so I talked about this example of what you want models to do when you're asking them to write 50 different evaluations.

所以我谈到了这个例子，当你要求模型写出 50 种不同的评估时，你希望模型做什么。

Do you just continue them for 50 more or do you just say, "No, just move on because this is perfect enough." And the broader question is, are we building these systems that actually advance humanity?

你是继续做 50 次以上，还是只是说：“不，继续吧，因为这已经足够完美了。”更广泛的问题是，我们正在构建这些真正推动人类进步的系统吗？

And if so how do we build the data sets to train towards that and measure it?

如果是这样，我们如何构建数据集来训练并测量它？

Are we optimizing for all of these wrong things, just systems that suck up more and more of our time and make us lazier and lazier?

我们是否正在针对所有这些错误的事情进行优化，只是那些占用我们越来越多时间并使我们变得越来越懒的系统？

Edwin Chen: And yeah, I think it's really relevant to what we do because it's very hard and difficult to measure and define whether something is genuinely advancing humanity.

Edwin Chen：是的，我认为这与我们所做的事情确实相关，因为衡量和定义某件事是否真正推动人类发展非常困难。

It's very easy to measure all these proxies instead like clicks and likes.

衡量所有这些代理非常容易，例如点击次数和点赞次数。

But I think that's why our work is so interesting.

但我认为这就是我们的工作如此有趣的原因。

We want to work the hard, important metrics that require the hardest types of data and not just the easy ones.

我们希望研究需要最困难的数据类型的重要指标，而不仅仅是简单的数据。

So I think one of the things I often say is you are your objective function.

所以我认为我经常说的一件事是你就是你的目标函数。

So we want the rich, complex, objective functions and not these simplistic proxies.

所以我们需要丰富、复杂的目标函数，而不是这些简单的代理。

And our job is to figure out how to get the data to match this.

我们的工作是弄清楚如何获得与此相匹配的数据。

Edwin Chen: So yeah, we want data, we want metrics that measure whether AI is making your life richer.

Edwin Chen：是的，我们需要数据，我们需要衡量人工智能是否让你的生活变得更加丰富的指标。

We want to train our systems this way.

我们希望以这种方式训练我们的系统。

And we want tools that make us more curious and more creative, not just lazier.

我们需要工具让我们更加好奇、更加富有创造力，而不仅仅是更加懒惰。

And it's hard because, yeah, humans are kind of inherently lazy so AI software deals are the easiest way to get engagement, make all your metrics fall up.

这很难，因为，是的，人类本质上是懒惰的，所以人工智能软件交易是获得参与度的最简单方法，让你的所有指标都下降。

So I think this question about choosing the right objective functions and making sure that we're optimizing towards them and not just these easy proxies is really important to our future.

因此，我认为关于选择正确的目标函数并确保我们针对它们进行优化而不仅仅是这些简单的代理的问题对于我们的未来非常重要。

Lenny Rachitsky: Wow.

莱尼·拉奇茨基：哇。

I love how what you're sharing here gives you so much more appreciation of the nuances of building AI, training AI, the work that you're doing.

我喜欢你在这里分享的内容，让你更加了解构建人工智能、训练人工智能以及你正在做的工作的细微差别。

Lenny Rachitsky: From the outside, people could just look at Surge and companies in the space of, okay, cool.

Lenny Rachitsky：从外部来看，人们可以在“好吧，很酷”的空间里看待 Surge 和公司。

They're just creating all this data, feeding it to AI.

他们只是创建所有这些数据，并将其输入人工智能。

But clearly there's so much to this that people don't realize, and I love knowing that you're at the head of this, that someone like you is thinking through this so deeply.

但显然，有很多事情人们没有意识到，我很高兴知道你是这件事的带头人，像你这样的人正在如此深入地思考这个问题。

Lenny Rachitsky: Maybe one more question, is there something you wish you'd known before you started Surge?

Lenny Rachitsky：也许还有一个问题，在开始 Surge 之前，您是否希望知道一些事情？

A lot of people start companies, they don't know what they're getting into.

很多人创办公司时，他们并不知道自己要做什么。

Is there something you wish you could tell your earlier self?

有什么事情你希望能告诉以前的自己吗？

Edwin Chen: Yeah, so I definitely wish I'd known that you could build a company by being heads down and doing great research and simply building something amazing.

Edwin Chen：是的，所以我绝对希望我知道你可以通过埋头苦干、进行大量研究并简单地构建一些令人惊叹的东西来建立一家公司。

And not by constantly tweeting and hyping and fundraising.

而不是通过不断发推文、炒作和筹款。

It's kind of funny, but I never thought I wanted to start a company.

这有点有趣，但我从未想过要创办一家公司。

I love doing research.

我喜欢做研究。

And I was actually always a huge fan of DeepMind because they were this amazing research company that got bought and still managed to keep on doing amazing science.

事实上，我一直是 DeepMind 的忠实粉丝，因为他们是一家令人惊叹的研究公司，被收购后仍然设法继续进行令人惊叹的科学研究。

But I always thought that they were this magical ILR unicorn.

但我一直以为他们就是这只神奇的ILR独角兽。

So I thought if I started a company, I'd have to become a business person looking at financials all day and being in meetings all day and doing all this stuff that sounded incredibly boring and I always hated.

所以我想，如果我创办一家公司，我就必须成为一名商人，整天关注财务，整天开会，做所有这些听起来非常无聊、我一直讨厌的事情。

So I think it's crazy that didn't end up being true at all.

所以我认为这很疯狂，但最终却根本不是真的。

I'm still in the weeds in the data every day.

我每天还在数据里杂草丛生。

And I love it.

我喜欢它。

I love that I get to do all these analyses and talk to researchers.

我喜欢能够进行所有这些分析并与研究人员交谈。

And it's basically applied research where we're building all these amazing data systems that have really pushed the frontier of AI.

它基本上是应用研究，我们正在构建所有这些令人惊叹的数据系统，这些系统真正推动了人工智能的前沿。

Edwin Chen: So yeah, I wish I know that you don't need to spend all your time fundraising.

Edwin Chen：是的，我希望我知道你不需要把所有的时间都花在筹款上。

You don't need to constantly generate hype.

你不需要不断地进行炒作。

You don't need to become someone you're not.

你不需要成为你不是的人。

You can actually build a successful company by simply building something so good that it cut through all that noise.

实际上，你可以通过简单地打造一些优秀的东西来消除所有噪音，从而打造一家成功的公司。

And I think if I known this was possible, I would've started even sooner, so I that.

我想如果我知道这是可能的，我会更早开始，所以我就这么做了。

Lenny Rachitsky: And that is such an amazing place to end.

Lenny Rachitsky：这真是一个令人惊奇的结局。

I feel like this is exactly what founders need to hear, and I think this conversation's going to inspire a lot of founders, and especially a lot of founders that want to do things in a different way.

我觉得这正是创始人需要听到的，我认为这次对话会激励很多创始人，尤其是很多想要以不同方式做事的创始人。

Before we get to a very exciting lightning round, is there anything else you wanted to share?

在我们开始激动人心的闪电回合之前，您还有什么想分享的吗？

Anything else you want to leave our listeners with?

您还有什么想留给我们的听众的吗？

We covered a lot of ground, it's totally okay to say no as well.

我们涵盖了很多内容，拒绝也是完全可以的。

Edwin Chen: I think the thing I would end with is I think a lot of people think of data labeling as it relates to simplistic work.

Edwin Chen：我认为我最后想说的是，很多人认为数据标签与简单的工作有关。

Like labeling cat photos and drawing bounding box around cars.

就像给猫的照片贴标签和在汽车周围绘制边界框一样。

And so I've actually always hated the word data labeling because it just paints this very simplistic picture when I think what we're doing is completely different.

所以我实际上一直讨厌数据标签这个词，因为当我认为我们正在做的事情完全不同时，它只是描绘了一幅非常简单的图画。

I think a lot about what we're doing as a lot more like raising a child.

我对我们正在做的事情思考了很多，就像抚养孩子一样。

You don't just feed a child information.

你不只是向孩子提供信息。

You're teaching them values, and creativity, and what's beautiful, and these infinite subtle things about what makes somebody a good person.

你正在教他们价值观、创造力、什么是美丽的，以及这些关于如何使一个人成为好人的无限微妙的东西。

And that's what we're doing for AI.

这就是我们为人工智能所做的事情。

So yeah, I just often think about what we're doing as almost like the future of humanity or how we're raising humanity's children, so I'll leave it at that.

所以，是的，我只是经常思考我们正在做的事情几乎就像人类的未来，或者我们如何抚养人类的孩子，所以我就到此为止。

Lenny Rachitsky: Wow.

莱尼·拉奇茨基：哇。

I love just how much philosophy there is in this whole conversation that I was not expecting.

我喜欢这整个谈话中蕴含的哲学，这是我没有预料到的。

Lenny Rachitsky: With that, Edwin, we've reached our very exciting lightning round, I've got five questions for you.

Lenny Rachitsky：埃德温，我们已经进入了非常激动人心的闪电回合，我有五个问题要问你。

Are you ready?

你准备好了吗？

Lenny Rachitsky: Here we go.

莱尼·拉奇茨基：我们开始吧。

What are two or three books that you find yourself recommending most to other people?

您发现自己最向其他人推荐的两三本书是什么？

Edwin Chen: Yes, so three books I often recommend are, first, Story of Real Life by Ted Chang.

Edwin Chen：是的，所以我经常推荐的三本书是，第一，Ted Chang的《真实生活的故事》。

It's my all time favorite short story and it's about a linguist learning and alien language, and I basically reread it every couple years.

这是我一直以来最喜欢的短篇故事，它是关于语言学家学习和外星语言的，我基本上每隔几年就会重读一次。

Lenny Rachitsky: And that's what the Interstellar was about?

Lenny Rachitsky：这就是《星际穿越》的意义所在？

Is that...

难道是……

Edwin Chen: Yeah, so there's a movie called Arrival...

Edwin Chen：是的，有一部电影叫《降临》……

Edwin Chen: ...

陈艾文：……

which was based off of the story,

这是根据故事改编的，

Edwin Chen: And then second, Myth of Sisyphus by Camus.

Edwin Chen：第二个是加缪的《西西弗斯神话》。

I actually can't really explain why I love this, but I always find a final chapter somehow are really inspiring.

我实际上无法真正解释为什么我喜欢这个，但我总是发现最后一章在某种程度上确实鼓舞人心。

Edwin Chen: And then third, Le Ton beau de Marot by Douglas Hofstadter.

Edwin Chen：然后是第三名，Douglas Hofstadter 的 Le Ton beau de Marot。

And so I think Gödel, Escher, Bach is his more famous book, but I've actually always loved this one better.

所以我认为哥德尔、埃舍尔、巴赫是他更著名的书，但实际上我一直更喜欢这本书。

It basically takes a single French poem and translates it 89 different ways and discusses all the motivations behind each translation.

它基本上以一首法国诗为基础，以 89 种不同的方式翻译它，并讨论了每个翻译背后的所有动机。

And so I've always loved the way it embodies this idea that translation isn't this robotic thing that you do.

所以我一直很喜欢它体现这一理念的方式，即翻译不是你所做的机器人一样的事情。

Instead, there's a million different ways to think about what makes a high quality translation, which makes a lot of ways I think about data and quality in LLMs.

相反，有一百万种不同的方式来思考什么是高质量的翻译，这使得我在法学硕士中思考数据和质量的方式多种多样。

Lenny Rachitsky: All these resonate so deeply with the way, with all the things we've been talking about, especially that first one, if that was your goal after school is like, "I want to help translate alien language." I'm not surprised you love that short story.

Lenny Rachitsky：所有这些都与我们一直在谈论的所有事情产生了深刻的共鸣，尤其是第一个，如果这是你放学后的目标，就像，“我想帮助翻译外星语言。”你喜欢那个短篇小说我并不感到惊讶。

Lenny Rachitsky: Next question, do you have a favorite recent movie or TV show you've really enjoyed?

Lenny Rachitsky：下一个问题，你最近有最喜欢的电影或电视节目吗？

Edwin Chen: One of my new all time favorite TV shows is something I found recently, it's called Travelers.

Edwin Chen：我最近发现的我最喜欢的电视节目之一，叫做《旅行者》。

It's basically about a group of travelers from the future who are sent back in time to prevent their .

它基本上是关于一群来自未来的旅行者被及时送回过去以阻止他们的。

Sorry, I just wrote that section.

抱歉，我刚刚写了那部分。

Edwin Chen: And then I actually just rewatched Contact, which is one of my all time favorite movies.

Edwin Chen：然后我实际上刚刚重看了《接触》，这是我一直以来最喜欢的电影之一。

So yeah, I think one of the things you'll notice about me is that, yeah, I love any kind of book or film that involves scientists deciphering alien communication.

所以，是的，我想你会注意到我的一件事是，是的，我喜欢任何涉及科学家破译外星通信的书籍或电影。

Again, just this dream I always had as a kid.

再说一次，这就是我小时候一直有的梦想。

That's so funny [inaudible 01:05:29].: Okay, is there a product you've recently discovered that you really love?

太有趣了 [听不清 01:05:29].：好的，您最近发现有一款您真正喜欢的产品吗？

Edwin Chen: So it's funny, but I was in SF earlier this week and I finally took Waymo for the first time.

Edwin Chen：这很有趣，但这周早些时候我在旧金山，我终于第一次使用了 Waymo。

Honestly, it was magical and it really felt like living in the future.

老实说，这很神奇，真的感觉就像生活在未来。

Lenny Rachitsky: Yeah, it's like the thing that...

Lenny Rachitsky：是的，这就像……

People hype it like crazy, but it always exceeds your expectations.

人们疯狂地炒作它，但它总是超出你的预期。

Edwin Chen: Yeah, it deserves the hype.

Edwin Chen：是的，它值得大肆宣传。

It was crazy.

太疯狂了。

Yeah, it's absurd.

是的，这很荒谬。

It's like, holy moly.

就像，神圣的钼。

If you're not in SF, you don't realize just how common these things are.

如果您不在旧金山，您不会意识到这些事情有多么普遍。

They're just all over the place.

它们到处都是。

Just driverless cars constantly going about, and when you go to an event at the end, there's just all these Waymos lined up picking people up.

无人驾驶汽车不断地行驶，当你最后去参加一个活动时，就会看到所有这些 Waymo 都在排队接人。

Yeah.

是的。

Waymo, good job.

韦莫，干得好。

Good job over there.: Do you have a favorite life motto that you find yourself coming back to in work or in life?

那边干得好。：您在工作或生活中是否有最喜欢的人生座右铭？

Edwin Chen: So I think I mentioned this idea that founders should build a company that only they could build.

Edwin Chen：所以我想我提到了这个想法，即创始人应该建立一家只有他们才能建立的公司。

Almost like it's this destiny that their entire life, and experiences, and interests shape them towards.

几乎就像他们的整个生活、经历和兴趣都在塑造着他们的命运一样。

And so I think that principle applies pretty broadly, not just the founders, but the people creating, I think.

因此，我认为这一原则适用范围相当广泛，不仅适用于创始人，而且适用于创造者。

Lenny Rachitsky: Well, let me follow that thread to unlightening this answer.

Lenny Rachitsky：好吧，让我按照这个线索来阐明这个答案。

Do you have any advice for how to build those sorts of experiences that help lead to that?

对于如何建立有助于实现这一目标的体验，您有什么建议吗？

Is it follow things that are interesting to you, because it's easy to say that, it's hard to actually acquire these really unique sets of experiences that allow you to create something really important?

它是否遵循你感兴趣的事情，因为说起来很容易，但实际上很难获得这些真正独特的经验，让你能够创造出真正重要的东西？

Edwin Chen: Yeah, so I think it would always be to really follow your interests and do what you love, and it's almost like a lot of decisions I make about Surge.

Edwin Chen：是的，所以我认为真正追随自己的兴趣并做自己喜欢的事情，这几乎就像我对 Surge 所做的很多决定一样。

I think one of the things that I didn't think about a couple years ago, but then someone said it to me, it's that companies in a sense are an embodiment of their CEO.

我认为几年前我没有想到的一件事是，公司在某种意义上是其首席执行官的化身。

And it's kind of funny.

这有点有趣。

I hadn't thought about that because I never quite knew what a CEO did.

我没有考虑过这一点，因为我从来不太了解首席执行官是做什么的。

I always thought a CEO was kind of generic and it's like, okay, you're just doing whatever VPs, and your board, and whatever, tell you to do and you're just saying yes to decisions.

我一直认为首席执行官是一种通用的人，就像，好吧，你只是做副总裁和董事会等告诉你做的事情，而你只是对决策说“是”。

But instead, it's this idea where when I think about certain big, hard decisions we have to make, I don't think what would the company do, I don't think what metrics are we trying to optimize, I just think, "What do I personally care about?

但相反，正是这个想法，当我考虑我们必须做出的某些重大、艰难的决定时，我不认为公司会做什么，我不认为我们要优化哪些指标，我只是想，“我个人关心什么？

What are my values?

我的价值观是什么？

And what do I want to see happen in the world?"

我希望看到世界上发生什么？”

Edwin Chen: And so I think following that idea about...

Edwin Chen：所以我认为遵循这个想法......

Okay, so ask yourself, what are the values you care about?

好吧，那么问问你自己，你关心的价值观是什么？

What are things you're trying to shape and not...

哪些东西是你想要塑造而不是...

What will look good on a dashboard?

什么在仪表板上看起来不错？

I think that results are pretty important.

我认为结果非常重要。

Lenny Rachitsky: I love how just you're just full of endless, beautiful, and very deep answers.

Lenny Rachitsky：我喜欢你充满无尽、美丽且非常深刻的答案。

Lenny Rachitsky: Final question.

莱尼·拉奇茨基：最后一个问题。

Something that you got quite famous for before starting Surge is you built this map while you were at Twitter that showed a map of the world and what people called, whether they called it soda or pop.

在创建 Surge 之前，你因在 Twitter 工作时制作了这张地图而闻名，它显示了世界地图以及人们的称呼，无论他们称之为苏打水还是流行音乐。

I don't know if it's called Soda Pop.

不知道是不是叫苏打水。

What was the name of this map?

这张地图的名字是什么？

Edwin Chen: Yeah, it was like the Soda Versus Pop dataset.

Edwin Chen：是的，这就像苏打水与汽水数据集。

Lenny Rachitsky: And so it's like a map of the United States and it tells you where people say pop versus soda, so do you say soda or pop?

Lenny Rachitsky：这就像一张美国地图，它告诉你人们在哪里说汽水和汽水，所以你说汽水还是汽水？

Edwin Chen: So I say soda, I'm a soda person.

Edwin Chen：所以我说汽水，我是一个汽水爱好者。

Lenny Rachitsky: Okay.

莱尼·拉奇茨基：好的。

And is that just like that's the right answer or it's like whatever you are, it's totally fine.

这就是正确的答案吗？或者无论你是谁，都没关系。

Edwin Chen: I think I'll look at you a little bit funny.

Edwin Chen：我想我会用有点滑稽的眼神看着你。

You say pop and I'll wonder where you came from, but I won't score on you too much.

你说流行，我会想知道你来自哪里，但我不会给你太多分数。

That's how I feel too.: Edwin, this was incredible.

我也是这么想的：埃德温，这太不可思议了。

This was such an awesome conversation.

这是一次非常棒的谈话。

I learned so much.

我学到了很多。

I think we're going to help a lot of people start their own companies, help their companies become more aligned with their values and just building better things.

我认为我们将帮助很多人创办自己的公司，帮助他们的公司更加符合他们的价值观，并创造更好的东西。

That's how I feel too.: Few final questions, where can folks find you online if they want to reach out?

这也是我的感受。：最后一个问题是，如果人们想联系您，可以在哪里找到您？

What roles are you hiring for?

您正在招聘哪些职位？

How can listeners be useful to you?

听众如何对你有用？

Edwin Chen: Yeah, so I used to love writing a blog, but I haven't had time in the past few years.

Edwin Chen：是的，所以我以前很喜欢写博客，但是这几年我没有时间。

But I am starting to write again, so definitely check out the Surge blog, surgehq.ai/blog, and yeah, hopefully I'll be running a lot more there.

但我又开始写作了，所以一定要看看 Surge 博客，surgehq.ai/blog，是的，希望我能在那里运行更多。

And I would say we're definitely always hiring, so for people who just love data and people who love this intersection of math, and language, and computer science, definitely reach out anytime.

我想说，我们肯定一直在招聘，所以对于那些热爱数据的人以及热爱数学、语言和计算机科学交叉点的人来说，绝对可以随时与我们联系。

Lenny Rachitsky: Awesome.

莱尼·拉奇茨基：太棒了。

And how can listeners be useful to you?

听众如何对你有用？

Is it just, I don't know, yeah, is there anything there?

只是，我不知道，是的，那里有什么吗？

Any asks?

有什么要问的吗？

Edwin Chen: So I would say definitely tell me blog topics that you like me to write about...

Edwin Chen：所以我想说一定要告诉我你喜欢我写的博客主题......

Edwin Chen: ...

陈艾文：……

and then I'm always fascinated by all of these AI failures that happen in the real world.

然后我总是对现实世界中发生的所有这些人工智能失败着迷。

So whenever you come across a really interesting failure that I think illustrates some deep question about how we want model to behave, there's just so many different ways a model can respond, I just oftentimes think there's just not a single right answer.

因此，每当你遇到一个非常有趣的失败时，我认为它说明了一些关于我们希望模型如何表现的深刻问题，模型可以做出多种不同的响应方式，我只是经常认为没有一个正确的答案。

And so whenever there's one of these examples, I just love seeing them.

因此，每当出现这样的例子时，我就喜欢看到它们。

Lenny Rachitsky: You need to share these on your blog.

Lenny Rachitsky：你需要在你的博客上分享这些。

I'm also...

我也...

I would love to see these.

我很想看到这些。

Lenny Rachitsky: Edwin, thank you so much for being here.

Lenny Rachitsky：Edwin，非常感谢你来到这里。

Bye everyone.: Thank you so much for listening.

大家再见：非常感谢您的聆听。

If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.

如果您发现这很有价值，您可以在 Apple Podcasts、Spotify 或您最喜欢的播客应用程序上订阅该节目。

Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast.

另外，请考虑给我们评分或留下评论，因为这确实有助于其他听众找到播客。

You can find all past episodes or learn more about the show at lennyspodcast.com.

您可以在 lennyspodcast.com 上找到所有过去的剧集或了解有关该节目的更多信息。

See you in the next episode.

下一集见。

中文为机器翻译，仅供参考。