OpenAI’s CPO on how AI changes must-have skills, moats, coding, startup playbooks, more | Kevin Weil
Kevin Weil: The AI models that you're using today is the worst AI model you will ever use for the rest of your life, and when you actually get that in your head, it's kind of wild.
Kevin Weil：你今天使用的人工智能模型是你余生中使用过的最糟糕的人工智能模型，当你真正意识到这一点时，它有点疯狂。
Everywhere I've ever worked before this, you kind of know what technology you're building on, but that's not true at all with AI.
在此之前我工作过的每个地方，你都知道自己正在构建什么技术，但对于人工智能来说却完全不是这样。
Every two months, computers can do something they've never been able to do before and you need to completely think differently about what you're doing.
每两个月，计算机就可以做一些以前从未做过的事情，你需要完全不同地思考你正在做的事情。
Lenny Rachitsky: You're chief product officer of maybe the most important company in the world right now.
Lenny Rachitsky：您可能是目前世界上最重要的公司的首席产品官。
I want to chat about what it's just like to be inside the center of the storm.
我想谈谈在风暴中心的感觉。
Kevin Weil: Our general mindset is in two months, there's going to be a better model and it's going to blow away whatever the current set of limitations are.
Kevin Weil：我们的总体想法是，两个月后，将会出现一个更好的模型，并且它将消除当前的限制。
And we say this to developers too.
我们也对开发人员这么说。
If you're building and the product that you're building is kind of right on the edge of the capabilities of the models, keep going because you're doing something right.
如果您正在构建并且您正在构建的产品有点处于模型功能的边缘，请继续前进，因为您正在做正确的事情。
Give it another couple months and the models are going to be great, and suddenly the product that you have that just barely worked is really going to sing.
再过几个月，模型就会变得很棒，突然之间，你刚刚勉强工作的产品真的会唱歌。
Lenny Rachitsky: Famously, you led this project at Facebook called Libra.
Lenny Rachitsky：众所周知，您在 Facebook 领导了这个名为 Libra 的项目。
Kevin Weil: Libra is probably the biggest disappointment of my career.
Kevin Weil：Libra 可能是我职业生涯中最大的失望。
It fundamentally disappoints me that this doesn't exist in the world today because the world would be a better place if we'd been able to ship that product.
令我从根本上感到失望的是，当今世界上不存在这种产品，因为如果我们能够运送该产品，世界将会变得更加美好。
We tried to launch a new blockchain.
我们尝试推出一个新的区块链。
It was a basket of currencies originally.
最初是一篮子货币。
It was integration into WhatsApp and Messenger.
它被集成到 WhatsApp 和 Messenger 中。
I would be able to send you 50 cents in WhatsApp for free.
我可以通过 WhatsApp 免费向您发送 50 美分。
It should exist.
它应该存在。
To be honest, the current administration is super friendly to crypto.
老实说，现任政府对加密货币非常友好。
Facebook's reputation is in a very different place.
Facebook 的声誉处于一个非常不同的位置。
Maybe they should go build it now.
也许他们现在应该去建造它。
Lenny Rachitsky: Today my guest is Kevin Weil.
莱尼·拉奇茨基：今天我的客人是凯文·韦尔。
Kevin is chief product officer at OpenAI, which is maybe the most important and most impactful company in the world right now, being at the forefront of AI and AGI and maybe someday super intelligence.
Kevin 是 OpenAI 的首席产品官，该公司可能是目前世界上最重要、最具影响力的公司，处于人工智能和通用人工智能（AGI）甚至有一天超级智能的前沿。
He was previously head of product at Instagram and Twitter.
他此前曾担任 Instagram 和 Twitter 的产品主管。
He was co-creator of the Libra Cryptocurrency at Facebook, which we chat about.
他是 Facebook 加密货币 Libra 的联合创始人，我们对此进行了讨论。
He's also on the boards of Planet and Strava and the Black Product Managers Network and the Nature Conservancy.
他还是 Planet 和 Strava 以及黑人产品经理网络和大自然保护协会的董事会成员。
He's also just a really good guy and he has so much wisdom to share.
他也是一个非常好的人，有很多智慧可以分享。
We chat about how OpenAI operates, implications of AI and how we will all work and build product, which markets within the AI ecosystem, companies like OpenAI won't likely go after and thus are good places for startups to own.
我们讨论了 OpenAI 的运作方式、人工智能的影响以及我们将如何工作和构建产品，以及人工智能生态系统中的哪些市场，像 OpenAI 这样的公司不太可能追随，因此是初创公司拥有的好地方。
Also, why learning the craft of writing evals is quickly becoming a core skill for product builders, what skills will matter most in an AI era and what he's teaching his kids to focus on and so much more.
此外，为什么学习编写评估的技巧正在迅速成为产品构建者的核心技能，哪些技能在人工智能时代最重要，以及他教孩子们要关注的内容等等。
Lenny Rachitsky: This is a very special episode and I am so excited to bring it to you.
Lenny Rachitsky：这是一个非常特别的剧集，我很高兴能把它带给你们。
If you enjoy this podcast, don't forget to subscribe and follow it in your favorite podcasting app or YouTube.
如果您喜欢这个播客，请不要忘记订阅并在您最喜欢的播客应用程序或 YouTube 中关注它。
If you become an annual subscriber of my newsletter, you get a year free of Perplexity Pro, Linear, Notion Superhuman and Granola.
如果您成为我的时事通讯的年度订阅者，您将获得一年免费的 Perplexity Pro、Linear、Notion Superhuman 和 Granola。
Check it out at lennysnewsletter.com and click bundle.
请访问 lennysnewsletter.com 查看并单击“捆绑”。
With that, I bring you Kevin Weil.
接下来，我为您带来凯文·威尔。
This episode is brought to you by Eppo.
本集由 Eppo 为您带来。
Eppo is a next-generation A/B testing and feature management platform built by alums of Airbnb and Snowflake for modern growth teams.
Eppo 是由 Airbnb 和 Snowflake 的校友为现代增长团队构建的下一代 A/B 测试和功能管理平台。
Companies like Twitch, Miro, ClickUp and DraftKings rely on Eppo to power their experiments.
Twitch、Miro、ClickUp 和 DraftKings 等公司依靠 Eppo 来支持他们的实验。
Experimentation is increasingly essential for driving growth and for understanding the performance of new features and Eppo helps you increase experimentation velocity while unlocking rigorous deep analysis in a way that no other commercial tool does.
实验对于推动增长和了解新功能的性能越来越重要，Eppo 可以帮助您提高实验速度，同时以其他商业工具无法做到的方式解锁严格的深度分析。
Lenny Rachitsky: When I was at Airbnb, one of the things that I loved most was our experimentation platform where I could set up experiments easily, troubleshoot issues, and analyze performance all on my own.
Lenny Rachitsky：当我在 Airbnb 时，我最喜欢的事情之一就是我们的实验平台，在这里我可以轻松地设置实验、解决问题并自行分析性能。
Eppo does all that and more with advanced statistical methods that can help you shave weeks off experiment time and accessible UI for diving deeper into performance and out-of-the-box reporting that helps you avoid annoying prolonged analytic cycles.
Eppo 通过先进的统计方法完成了这一切，并提供了更多功能，可以帮助您缩短数周的实验时间，并且可访问的 UI 可以更深入地了解性能，而开箱即用的报告可以帮助您避免烦人的长时间分析周期。
Eppo also makes it easy for you to share experiment insight with your team, sparking new ideas for the A/B testing flywheel.
Eppo 还使您可以轻松地与您的团队分享实验见解，激发 A/B 测试飞轮的新想法。
Eppo powers experimentation across every use case, including product, growth, machine learning, monetization, and email marketing.
Eppo 为每个用例提供实验支持，包括产品、增长、机器学习、货币化和电子邮件营销。
Check out Eppo at geteppo.com/lenny and 10X your experiment velocity.
请访问 geteppo.com/lenny 查看 Eppo，将您的实验速度提高 10 倍。
That's geteppo.com/lenny.
那是 geteppo.com/lenny。
Lenny Rachitsky: This episode is brought to you by Persona, the adaptable identity platform that helps businesses fight fraud, meet compliance requirements, and build trust.
Lenny Rachitsky：本集由 Persona 为您带来，Persona 是一个适应性强的身份平台，可帮助企业打击欺诈、满足合规性要求并建立信任。
While you're listening to this right now, how do you know that you're really listening to me, Lenny?
当你现在正在听这个时，你怎么知道你真的在听我说话，莱尼？
These days, it's easier than ever for fraudsters to steal PII, faces and identities.
如今，欺诈者比以往任何时候都更容易窃取 PII、面孔和身份。
That's where Persona comes in.
这就是角色发挥作用的地方。
Persona helps leading companies like LinkedIn, Etsy, and Twilio securely verify individuals and businesses across the world.
Persona 帮助 LinkedIn、Etsy 和 Twilio 等领先公司安全地验证世界各地的个人和企业。
What sets Persona apart is its configurability.
Persona 的独特之处在于它的可配置性。
Every company has different needs depending on its industry, use cases, risk tolerance and user demographics.
每家公司根据其行业、用例、风险承受能力和用户人口统计数据都有不同的需求。
That's why Persona offers flexible building blocks that allow you to build tailored collection and verification flows that maximize conversion while minimizing risks.
这就是为什么 Persona 提供灵活的构建模块，使您能够构建定制的收集和验证流程，从而最大限度地提高转化率，同时最大限度地降低风险。
Plus Persona's orchestration tools automate your identity process so that you can fight rapidly shifting fraud and meet new waves of regulation.
Plus Persona 的编排工具可自动化您的身份流程，以便您能够对抗快速变化的欺诈行为并满足新一波的监管要求。
Whether you're a startup or an enterprise business, Persona has a plan for you.
无论您是初创公司还是大型企业，Persona 都有适合您的计划。
Learn more at withpersona.com/lenny.
如需了解更多信息，请访问 withpersona.com/lenny。
Again, that's withpersona.com/lenny.
同样，这是 withpersona.com/lenny。
Kevin, thank you so much for being here and welcome to the podcast.
凯文，非常感谢您来到这里，欢迎来到播客。
Kevin Weil: Thank you so much for having me.
凯文·韦尔：非常感谢您邀请我。
We've been talking about doing this forever and we made it happen.
我们一直在谈论这样做，并且我们做到了。
Lenny Rachitsky: We did it.
莱尼·拉奇茨基：我们做到了。
I can't imagine how insane your life is, so I really appreciate that you made time for this and we're actually recording this the week that you guys launched your new image model, which is a happy coincidence.
我无法想象你们的生活有多么疯狂，所以我真的很感谢你们抽出时间来做这件事，我们实际上是在你们推出新形象模型的那一周录制的，这是一个令人高兴的巧合。
My entire social feed is filled with ghiblifications of everyone's life and family photos and everything, so good job.
我的整个社交动态充满了每个人的生活和家庭照片以及一切的吉利化，干得好。
Kevin Weil: Yep, mine too.
凯文·威尔：是的，我的也是。
My wife, Elizabeth, sent me one of hers, so I'm right there with you.
我的妻子伊丽莎白给我寄了一件，所以我就在你身边。
Lenny Rachitsky: Let me just ask, did you guys expect this kind of reaction?
Lenny Rachitsky：我想问一下，你们预料到会出现这样的反应吗？
It feels like this is the most viral thing that's happened in AI, which a high bar since, I don't know, ChatGPT launched.
感觉这是人工智能领域发生的最具病毒性的事情，自 ChatGPT 推出以来，这是一个很高的标准。
Just like, did you guys expect it to go this well?
就像，你们期望事情会这么顺利吗？
What does it feel like internally?
内心感觉如何？
Kevin Weil: There have been a handful of times in my career when you're working on a product internally and the internal usage just explodes.
Kevin Weil：在我的职业生涯中，有过几次当你在内部开发产品时，内部使用量就会激增。
This was true by the way when we were building stories at Instagram.
顺便说一句，当我们在 Instagram 上构建故事时，情况确实如此。
More than anything else in my career, we could feel it was going to work because we were all using it internally and we'd go away for a weekend.
在我的职业生涯中，最重要的是，我们能感觉到它会起作用，因为我们都在内部使用它，而且我们会出去度一个周末。
Before it launched we were all using it and we'd come back after a weekend and we would know what was going on and be like, "Oh, hey, I saw you were at that camping trip, how was that?" You were like, "Man, this thing really works." ImageGen was definitely one of those, so we'd been playing with it for, I don't know, a couple months and when it first went live internally to the company, there was kind of a little gallery where you could generate your own, you could also see what everyone else was generating and it was just nonstop buzz.
在它推出之前，我们都在使用它，周末后我们回来，我们会知道发生了什么，然后会说，“哦，嘿，我看到你参加了那个露营旅行，怎么样？”你会说，“伙计，这东西真的有用。” ImageGen 绝对是其中之一，所以我们已经使用它几个月了，我不知道，当它第一次在公司内部上线时，有一个小画廊，你可以在其中生成自己的，你也可以看到其他人在生成什么，这只是不间断的嗡嗡声。
So yeah, we had a sense that this was going to be a lot of fun for people to play with.
所以，是的，我们有一种感觉，这对人们来说会很有趣。
Lenny Rachitsky: That's really cool.
莱尼·拉奇茨基：这真的很酷。
That should be a measure of just confidence into something going well that you're launching is internally everyone's going crazy for it.
这应该是对你正在推出的事情进展顺利的信心的衡量标准，因为内部每个人都为之疯狂。
Kevin Weil: Yeah.
凯文·威尔：是的。
Especially social things because you have a very tight network as a company socially, so you know each other and you're experts in your product hopefully.
尤其是社交方面，因为作为一家公司，你们拥有非常紧密的社交网络，所以你们彼此了解，并且有望成为产品方面的专家。
And so there's some sense in which if you're doing something social and it's not taking off internally, you might question what you're doing.
因此，从某种意义上说，如果你正在做一些社交活动，但它在内部没有取得成功，你可能会质疑自己在做什么。
Lenny Rachitsky: Yeah, and by the way, the Ghibli thing, is that something you guys seeded or how did that even start?
Lenny Rachitsky：是的，顺便说一句，吉卜力的事情，是你们播种的吗？或者它是如何开始的？
Was that an intentional example?
这是故意的例子吗？
Kevin Weil: I think it's just the style people love and the model is really capable at emulating style or understanding what...
Kevin Weil：我认为这正是人们喜爱的风格，而模特确实有能力模仿风格或理解什么......
It's very good at instruction following.
它非常善于遵循指令。
That's actually something that I think people...
这实际上是我认为人们...
I'm starting to see people discover with it, but you can do very complex things.
我开始看到人们用它来发现，但你可以做非常复杂的事情。
You can give it two images, one is your living room and the other is a whole bunch of photos or memorabilia or things you want and you say, "Tell me how you would arrange these things." Or you can say, "I'd like you to show me what this will look like if you put this over here and this thing to the right of that and this one to the left of this, but under that one." And the model actually will understand all of that and do it.
你可以给它两张图片，一张是你的客厅，另一张是一大堆照片、纪念品或你想要的东西，然后你说，“告诉我你会如何安排这些东西。”或者你可以说，“我想让你告诉我，如果你把这个放在这里，这个东西放在那个东西的右边，这个东西放在这个东西的左边，但在那一个东西下面，这会是什么样子。”该模型实际上会理解所有这些并做到这一点。
It's incredibly powerful.
它非常强大。
So I'm just excited about all the different things people are going to figure out.
所以我对人们将要弄清楚的所有不同的事情感到兴奋。
Lenny Rachitsky: Yeah.
莱尼·拉奇茨基：是的。
All right.
好的。
Well, good job.
嗯，干得好。
Good job team OpenAI.
OpenAI 团队干得好。
Let's get serious here and let's zoom out a little bit.
让我们严肃一点，然后把范围缩小一点。
The way I see it is you're chief product officer of maybe the most important company in the world right now.
我的看法是，你可能是目前世界上最重要的公司的首席产品官。
Just not to set the bar too high, but you guys are ushering in AI, AGI at some point, super intelligence at some point.
只是不要把标准定得太高，但你们在某个时候会迎来人工智能、通用人工智能，在某个时候会迎来超级智能。
No big deal.
没什么大不了的。
I have more questions for you than I've had for any other guest.
我向您提出的问题比向其他房客提出的问题还要多。
Actually put out a call-out on Twitter and LinkedIn and my community just like what would you want to ask Kevin?
实际上在 Twitter、LinkedIn 和我的社区上发出呼吁，就像你想问凯文什么？
And I had over 300 well-formed questions and we're going to go through every single one.
我提出了 300 多个结构良好的问题，我们将逐一逐一进行讨论。
So let's just get started.
那么让我们开始吧。
I'm just joking.
我只是开玩笑。
Lenny Rachitsky: I picked out the best and there's a lot of stuff I'm really curious about.
Lenny Rachitsky：我挑选了最好的，其中有很多我真的很好奇的东西。
Kevin Weil: Well, it's 1 PM here.
Kevin Weil：嗯，现在是下午 1 点。
It doesn't get dark for a while, so let's do it.
天一时还没有黑，咱们就动手吧。
Lenny Rachitsky: Okay, here we go.
莱尼·拉奇茨基：好的，我们开始吧。
Okay, so first of all, I'm just going to take notes here.
好的，首先，我要在这里做笔记。
When is AGI launching?
AGI什么时候上线？
When in December?
十二月什么时候？
Kevin Weil: I mean, we just launched a good ImageGen model.
Kevin Weil：我的意思是，我们刚刚推出了一个很好的 ImageGen 模型。
Does that count?
这算吗？
Lenny Rachitsky: It's getting there.
Lenny Rachitsky：快到了。
It's getting there.
快到了。
Kevin Weil: There's this quote I love, which is "AI is whatever hasn't been done yet" because once it's been done, when it kind of works, then you call it machine learning, and once it's kind of ubiquitous and it's everywhere, then it's just an algorithm.
凯文·威尔：我喜欢这句话，“人工智能是尚未完成的事情”，因为一旦它完成了，当它起作用时，你就可以称之为机器学习，而一旦它变得无处不在，无处不在，那么它就只是一种算法。
So I've always loved that we call things AI when they still don't quite work and then by the time it's like an AI algorithm that's recommending you follow, oh, that's just an algorithm, but this new thing, like self-driving cars, that's it.
所以我一直喜欢当它们仍然不太有效时我们称之为人工智能，然后当它就像一个人工智能算法建议你遵循时，哦，这只是一个算法，但这个新事物，比如自动驾驶汽车，就是这样。
I think to some degree we're always going to be there and the next thing is always going to be AI and the current thing that we use every day and is just a part of our lives, that's an algorithm.
我认为在某种程度上，我们总是会在那里，下一件事总是人工智能，而我们每天使用的当前事物只是我们生活的一部分，那就是算法。
Lenny Rachitsky: It's so interesting because in the Bay Area you see self-driving cars driving around and it's so normal now when four years ago and three years ago, you would've seen this and you'd be like, "Holy shit, what is...
Lenny Rachitsky：这很有趣，因为在湾区你会看到自动驾驶汽车到处行驶，现在这很正常，四年前和三年前，你会看到这个，然后你会说，“天哪，这是什么......
We're in the future." And now we're just so take it for granted.
我们就在未来。”现在我们只是认为这是理所当然的。
Kevin Weil: I mean there's something like that with everything.
凯文·威尔：我的意思是，一切事物都存在类似的情况。
If I showed you...
如果我给你看...
When GPT-3 launched, I wasn't at OpenAI then.
当 GPT-3 推出时，我当时并不在 OpenAI。
I was just a user, but it was mind-blowing.
我只是一个用户，但这令人兴奋。
And if I gave you GPT-3 now I just plugged that into ChatGPT for you and you started using it, you'd be like, "What is this thing?" It's like mess.
如果我现在给你 GPT-3，我只是为你将其插入 ChatGPT，然后你开始使用它，你会说，“这是什么东西？”就像一团糟。
Kevin Weil: I had the same experience when I first got into a Waymo, your very first ride, at least my very first ride, my first 10 seconds in a Waymo, it starts driving and you're like, "Oh my God, watch out for that bike." You're holding onto whatever you can.
Kevin Weil：当我第一次骑上 Waymo 时，我也有同样的经历，你的第一次骑行，至少是我的第一次骑行，我在 Waymo 上的前 10 秒，它开始行驶，你会说，“天哪，小心那辆自行车。”你正在抓住一切你能抓住的东西。
And then five minutes in, you've calmed down and you realize that you're getting driven around the city without a driver and it's working.
五分钟后，你平静下来，你意识到你在没有司机的情况下在城市里开车，而且它正在发挥作用。
You're just like, "Oh my God, I am living in the future right now." And then another 10 minutes, you're bored, you're doing email on your phone, answering Slack messages, and suddenly this miracle of human invention is just an expected part of your life from then on.
你就像是，“天啊，我现在就生活在未来。”然后又过了 10 分钟，你感到无聊，你在手机上写电子邮件，回复 Slack 消息，突然间，这个人类发明的奇迹就成了你生活中期待的一部分。
And there is really something in the way that we all are adapting to AI that's kind of like that.
我们所有人都在适应人工智能，确实存在类似的情况。
These miraculous things happen and computers can do something they've never been able to do before and it blows our mind collectively for a week and then we're like, oh, yeah.
这些奇迹般的事情发生了，计算机可以做一些他们以前从未做过的事情，这让我们集体震惊了一周，然后我们就像，哦，是的。
Oh, now it's just machine learning on its way to being an algorithm.
哦，现在机器学习正在成为一种算法。
Lenny Rachitsky: The craziest thing about what you just shared actually is, I don't know, ChatGPT, which is now feels terrible.
Lenny Rachitsky：你刚刚分享的内容中最疯狂的事情实际上是，我不知道，ChatGPT，现在感觉很糟糕。
3.5 was a couple years ago, and imagine what life will be like in a couple years from now.
3.5 是几年前的事了，想象一下几年后的生活会是什么样子。
We're going to get to that, where things are going, what you think is going to be the next big leap.
我们将讨论事情的进展情况，以及你认为的下一个重大飞跃。
But I want to start with the beginning of your journey at OpenAI.
但我想从您在 OpenAI 的旅程开始。
So you worked at Twitter, you worked at Facebook, you worked at Planet, Instagram.
所以你在 Twitter 工作过，你在 Facebook 工作过，你在 Planet 工作过，Instagram 工作过。
At some point you got recruited to go and come work at OpenAI.
在某个时候，你被招募到 OpenAI 工作。
I'm curious just what that story was like of the recruiting process of joining OpenAI as CPO.
我很好奇加入 OpenAI 作为 CPO 的招聘过程是怎样的。
Is there any fun stories there?
这里面有什么有趣的故事吗？
Kevin Weil: If I'm remembering the timeline right, we communicated at Planet I was leaving and I was planning to just go take some time.
凯文·威尔：如果我没记错时间线的话，我们在 Planet 上沟通过我要离开，我打算去花一些时间。
I wasn't going to stop working, but I was also happy to take the summer.
我不会停止工作，但我也很高兴度过这个夏天。
This was maybe April or something.
这可能是四月之类的。
I was like, cool, I'm going to have the summer with my kids.
我当时想，太酷了，我要和我的孩子们一起度过夏天。
We're going to go to Tahoe or something and I'll actually get to hang out rather than what I usually do going up and down and all that.
我们要去太浩湖之类的地方，我实际上可以出去玩，而不是像我通常做的那样上上下下之类的。
And then Sam and I had known each other lightly for a bunch of years and he's always involved in so many interesting things like companies building fusion and all these things.
然后萨姆和我已经认识很多年了，他总是参与很多有趣的事情，比如公司建立融合和所有这些事情。
So he'd always been somebody that I would call occasionally if I was starting to think about my next thing because I like working on big tech forward, sort of next wave kind of things.
因此，如果我开始考虑我的下一步工作，我偶尔会打电话给他，因为我喜欢从事大型科技前沿工作，类似于下一波的事情。
Kevin Weil: And so I called him and I think Vinod also helped to put us in touch again.
凯文·韦尔：所以我给他打了电话，我认为维诺德也帮助我们再次取得了联系。
And this time it wasn't like, "Oh, you should go talk to these guys working on fusion." He said, "Actually, we're thinking about something, you should come talk to us." I was like, "Okay, that sounds amazing.
这次不再是“哦，你应该去和这些从事核聚变研究的人谈谈。”他说：“其实我们正在考虑一些事情，你应该过来和我们谈谈。”我当时想，“好吧，这听起来棒极了。
Let's do it." And it goes really fast, really, really fast.
让我们开始吧。”而且进展得非常快，非常非常快。
I met most of the management team in a brief period of time, a few days, and they were telling me, 'Look, we're basically going to move as fast as we want to move.
我在短短几天内会见了大多数管理团队的成员，他们告诉我，‘看，我们基本上会按照我们想要的速度采取行动。
And if you talk to everyone, everyone likes you, you're ready to go." Sam came over for dinner and we had a great evening together just talking about OpenAI in the future and getting to know each other better.
如果你和每个人交谈，每个人都喜欢你，你就可以出发了。”Sam 过来吃晚饭，我们一起度过了一个愉快的夜晚，谈论未来的 OpenAI 并更好地了解彼此。
And at the end I was like, I was going to go in the next day for a bigger round of interviews and Sam was saying, "Hey, it's going really well.
最后我想，第二天我要去参加一轮更大的面试，萨姆说，“嘿，进展非常顺利。
We're really excited."
我们真的很兴奋。”
Kevin Weil: And I said, "Cool.
凯文·韦尔：我说，“酷。
So how do I think about tomorrow?" And he said, "Oh, you'll be fine.
那么我该如何考虑明天呢？”他说，“哦，你会没事的。
Don't worry about it.
别担心。
And if it goes well, we're basically there." And so I go in the next day, meet a bunch of people, have a great time.
如果进展顺利，我们基本上就到了。”所以我第二天就去了，遇到了很多人，度过了一段愉快的时光。
I really enjoyed everybody I met with.
我真的很喜欢我遇到的每一个人。
In any interview, you can always second guess yourself like, oh, I shouldn't have said that thing or that thing I gave a bad answer on I wish I could redo, but I came away feeling like I think that went pretty well.
在任何采访中，你总是可以重新审视自己，比如，哦，我不应该说那件事或那件事，我给出了一个糟糕的答案，我希望我可以重做，但我离开时感觉我认为一切进展顺利。
And I was expecting to hear that weekend basically because they sort of set expectations as soon as if this goes well, we're ready to go.
我期待那个周末的消息，基本上是因为他们设定了期望，一旦进展顺利，我们就准备好了。
And I didn't hear anything.
我什么也没听到。
And then it was like Monday, Tuesday, Wednesday, I still didn't hear anything and I reached out to folks on the OpenAI side a couple of times, still nothing.
然后就像周一、周二、周三，我仍然没有听到任何消息，我联系了 OpenAI 方面的人员几次，仍然没有任何消息。
Kevin Weil: And I was like, "Oh my God, I screwed it up.
凯文·韦尔：我当时想，“天哪，我搞砸了。
I don't know where I screwed it up, but I totally screwed it up.
我不知道我哪里搞砸了，但我彻底搞砸了。
I can't believe it." And I was going back to Elizabeth, my wife and being like, "What did I do?
我简直不敢相信。”我回到我的妻子伊丽莎白身边，心想：“我做了什么？
Where do you think I..." Getting all crazy about it and then it's still nothing.
你认为我在哪里……”变得疯狂，然后仍然什么也没有。
And finally it was like nine days later, they finally got back to me and it turned out there was a bunch of stuff happening internally and this, that and the other thing, and there's just a million things happening.
最后，就像九天后，他们终于回到我身边，结果发现内部发生了很多事情，这个，那个和其他的事情，而且只有一百万件事正在发生。
And they finally were like, "Oh yeah, that went well.
他们最后说：“哦，是的，进展顺利。
Let's do this." And I was like, "Oh, okay, cool, let's do it." But it was nine days of agony and they were just super busy on some internal stuff and there I was fretting every single day and re-going over every line of our interview process.
我们就这么做吧。”我当时想，“哦，好吧，酷，我们就这么做吧。”但那是九天的痛苦，他们只是非常忙于一些内部事务，而我每天都在烦恼，并重新审视我们面试过程的每一行。
Lenny Rachitsky: It makes me think about when you're dating someone and you've texted them and you're not hearing anything back, you assume something is wrong.
Lenny Rachitsky：这让我想到，当你和某人约会时，你给他们发短信，但没有收到任何回复，你就会认为出了什么问题。
Kevin Weil: I have a hard time about it still.
凯文·威尔：我仍然很难接受。
Lenny Rachitsky: That's wild.
莱尼·拉奇茨基：这太疯狂了。
I love that it worked out.
我喜欢它成功了。
And I guess the lesson there is don't jump to conclusions.
我想这里面的教训是不要急于下结论。
Kevin Weil: Yeah.
凯文·威尔：是的。
Have a little bit of chill.
稍微冷静一下。
Lenny Rachitsky: Speaking of that, I want to chat about what it's just like to be inside the center of the storm.
Lenny Rachitsky：说到这里，我想谈谈身处风暴中心的感觉。
Again, you work at a lot of, let's say traditional companies even though they're not that traditional, Twitter and Instagram and Facebook and Planet, and now you work at OpenAI.
再说一次，你在很多传统公司工作，尽管它们并不那么传统，比如 Twitter、Instagram、Facebook 和 Planet，现在你在 OpenAI 工作。
I'm curious, what is most different about how things work in your day-to-day life at OpenAI?
我很好奇，您在 OpenAI 的日常生活中最不同的是什么？
Kevin Weil: I think it's probably the pace.
凯文·威尔：我认为可能是节奏。
Maybe it's two things.
也许这是两件事。
One is it's the pace.
一是节奏。
The second is everywhere I've ever worked before this, you kind of know what technology you're building on.
第二个是我之前工作过的任何地方，你都知道你正在构建什么技术。
So you spend your time thinking about what problems are you solving?
所以你花时间思考你要解决什么问题？
Who are you building for?
你为谁建造？
How are you going to make their lives better?
你将如何让他们的生活变得更好？
How are you going to...
你要怎样...
Is this a big enough problem that you're going to be able to change habits?
这是一个足够大的问题，你能够改变习惯吗？
Do people care about this problem being solved?
人们关心这个问题是否得到解决吗？
All those good product things.
所有这些好产品。
But the stuff that you're building on is kind of fixed.
但你所构建的东西是固定的。
You're talking about databases and things and I bet the database you used this year is probably 5% better than the database you used two years ago, but that's not true at all with AI.
你谈论的是数据库之类的东西，我敢打赌你今年使用的数据库可能比两年前使用的数据库好 5%，但对于人工智能来说根本不是这样。
It's like every two months computers can do something they've never been able to do before and you need to completely think differently about what you're doing.
就像每两个月计算机就可以做一些他们以前从未做过的事情，你需要完全不同地思考你正在做的事情。
Kevin Weil: There's something fundamentally interesting about that makes life fun here.
凯文·威尔：有一些本质上有趣的东西让这里的生活变得有趣。
There's also something we will maybe talk about evals later, but it also really, in this world of...
还有一些事情我们稍后可能会讨论评估，但它也确实在这个世界上......
Everything we're used to with computers is about giving a computer very defined inputs.
我们对计算机所习惯的一切都是为计算机提供非常明确的输入。
If you look at Instagram for example, there are buttons that do specific things and you know what they do.
例如，如果您查看 Instagram，就会发现有些按钮可以执行特定操作，并且您知道它们的作用。
And then when you give a computer defined inputs, you get very defined outputs.
然后，当你给计算机定义输入时，你会得到非常定义的输出。
You're confident that if you do the same thing three times, you're going to get the same output three times.
你确信如果你做同样的事情三次，你将会得到同样的输出三次。
LLMs are completely different than that.
LLM 与此完全不同。
They're good at fuzzy subtle inputs.
他们擅长模糊微妙的输入。
Then all the nuances of human language and communication, they're pretty good at.
然后，他们非常擅长人类语言和交流的所有细微差别。
And also they don't really give you the same answer.
而且他们并没有真正给你相同的答案。
You probably get spiritually the same answer for the same question, but it's certainly not the same set of words every time.
对于同一个问题，你可能在精神上得到相同的答案，但每次肯定不是同一组词。
And so you're much more, it's fuzzier inputs and fuzzier outputs.
所以你还有更多，它是更模糊的输入和更模糊的输出。
And when you're building products, it really matters whether there's some use case that you're trying to build around.
当您构建产品时，是否存在您尝试构建的用例确实很重要。
Kevin Weil: If the model gets it right 60% of the time, you build a very different product than if the model gets it right 95% of the time versus if the model gets it right 99.5% of the time.
Kevin Weil：如果模型在 60% 的情况下正确，那么您构建的产品与模型在 95% 的情况下正确的情况与模型在 99.5% 的情况下正确的情况截然不同。
And so there's also something that you have to get really into the weeds on your use case and the evals and things like that in order to understand the right kind of product to build.
因此，为了了解要构建的正确产品类型，您还必须真正深入了解用例和评估以及类似的事情。
So that is just fundamentally different.
所以这是根本不同的。
If your database works once, it works every time.
如果您的数据库工作一次，那么它每次都工作。
And that's not true in this world.
但在这个世界上情况并非如此。
Lenny Rachitsky: Let's actually follow this thread on evals.
Lenny Rachitsky：让我们实际上关注这个评估线索。
I definitely wanted to talk about this.
我确实想谈谈这个。
We had this legendary panel at the Lenny & Friends Summit.
我们在 Lenny & Friends 峰会上举行了这个传奇的小组讨论。
It was you and Mike Krieger and Sarah Guo moderating.
由你、迈克·克里格和莎拉·郭主持。
Lenny Rachitsky: So fun.
莱尼·拉奇茨基：太有趣了。
The thing that I heard that kind of stuck with people from that panel was a comment you made where you said that writing evals is going to become a core skill for product managers, and I feel like that probably applies further than just product managers.
我听到那个小组中的人们对你的评论感到困惑，你说编写评估将成为产品经理的核心技能，我觉得这可能不仅仅适用于产品经理。
A lot of people know what evals are.
很多人都知道 eval 是什么。
A lot of people have no idea what I'm talking about.
很多人不知道我在说什么。
So could you just briefly explain what is an eval and then just why do you think this is going to be so important for people building products in the future?
那么您能否简单解释一下什么是评估，然后您认为为什么这对于未来构建产品的人们如此重要？
Kevin Weil: Yeah, sure.
凯文·威尔：是的，当然。
I think the easiest way to think about it is almost like a quiz for a model, a test to gauge how well it knows a certain set of subject material or how good it is at responding to a certain set of questions.
我认为最简单的思考方式几乎就像是对模型的测验，一种测试，以衡量它对一组特定主题材料的了解程度或它回答一组特定问题的能力。
So in the same way you take a calculus class and then you have calculus tests that see if you've learned what you're supposed to learn.
因此，以同样的方式，您参加微积分课程，然后进行微积分测试，看看您是否学到了应该学习的内容。
You have evals that test how good is the model at creative writing?
您有评估来测试模型在创意写作方面的表现如何？
How good is the model at graduate level science?
该模型在研究生科学水平上有多好？
How good is the model at competitive coding?
该模型在竞争性编码方面的表现如何？
And so you have these set of evals that basically perform as benchmarks for how smart or capable the model is.
因此，您拥有这些评估集，它们基本上可以作为模型智能程度或能力的基准。
Lenny Rachitsky: Is it a simple way to think about it, like unit tests for model?
Lenny Rachitsky：这是一种简单的思考方式吗，就像模型的单元测试一样？
Kevin Weil: Yeah, unit tests, tests in general for models.
Kevin Weil：是的，单元测试，模型的一般测试。
Totally.
完全。
Lenny Rachitsky: Great, great.
莱尼·拉奇茨基： 太棒了，太棒了。
Okay.
好的。
And then why is this so important for people that don't totally understand what the hell's going on here with evals?
那么为什么这对于那些不完全理解 eval 到底发生了什么的人来说如此重要呢？
Why is this so key to building AI products?
为什么这对于构建人工智能产品如此关键？
Kevin Weil: Well, it gets back to what I was saying.
凯文·韦尔：好吧，这又回到了我刚才所说的。
You need to know whether your model is going to...
您需要知道您的模型是否会...
There are certain things that models will get right.
模型会正确处理某些事情。
99.95% of the time and you can just be confident.
99.95% 的情况下，您都可以充满信心。
There are things that they're going to be 95% right on and things they're going to be 60% right on.
有些事情他们的正确率是 95%，有些事情他们的正确率是 60%。
If the model's 60% right on something, you're going to need to build your product totally differently.
如果模型在某件事上有 60% 的正确率，那么您将需要以完全不同的方式构建您的产品。
And by the way, these things aren't static either.
顺便说一句，这些东西也不是静态的。
So a big part of evals is if you know you're building for some use case.
因此，评估的一个重要部分是您是否知道自己正在为某些用例进行构建。
So let's take our deep research product, which is one of my favorite things that we've released maybe ever.
让我们来看看我们的深度研究产品，这可能是我们发布过的我最喜欢的产品之一。
The idea is with deep research for people who haven't used it, you can give ChatGPT now an arbitrarily complex query.
这个想法是通过对没有使用过它的人进行深入研究，你现在可以给 ChatGPT 一个任意复杂的查询。
It's not about returning you an answer from a search query, which we can also do.
这并不是要从搜索查询中返回答案，而我们也可以做到这一点。
Kevin Weil: It's here's a thing that if you were going to answer it yourself, you'd go off and do two hours of reading on the web and then you might need to read some papers and then you would come back and start writing up your thoughts and realize you had some gaps in your thinking.
凯文·韦尔：如果你要自己回答这个问题，你会出去花两个小时在网上阅读，然后你可能需要阅读一些论文，然后你会回来开始写下你的想法，并意识到你的想法存在一些差距。
So you go out and do more research.
所以你出去做更多研究。
It might take you a week to write some 20 page answer to this question.
您可能需要一周的时间才能写出这个问题大约 20 页的答案。
You can let ChatGPT just like chug for you for 25, 30 minutes.
您可以让 ChatGPT 就像 chug 一样为您服务 25、30 分钟。
It's not the immediate answers you're used to, but it might go work for 25, 30 minutes and do work that would've taken you a week.
这不是您习惯的立即答案，但可能需要 25、30 分钟才能完成需要一周才能完成的工作。
So as we were building that product, we were designing evals at the same time as we were thinking about how this product was going to work and we were trying to go through hero use cases.
因此，当我们构建该产品时，我们在设计评估的同时，思考该产品将如何工作，并尝试浏览英雄用例。
Kevin Weil: Here's a question you want to be able to ask.
凯文·威尔：这是一个你想问的问题。
Here's an amazing answer for that question.
这是这个问题的一个令人惊奇的答案。
And then turning those into evals and then hill climbing on those evals.
然后将它们转化为评估，然后在这些评估上进行爬山。
So it's not just that the model is static and we hope it does okay on a certain set of things, you can teach the model.
因此，不仅仅是模型是静态的，我们希望它在某些事情上表现良好，你可以教授模型。
You can make this a continuous learning process.
您可以将其作为一个持续学习的过程。
And so as we were fine-tuning our model for deep research to be able to answer these things, we were able to test is it getting better on these evals that we said were important measures of how the product was working?
因此，当我们微调我们的模型以进行深入研究以能够回答这些问题时，我们能够测试这些评估是否变得更好，我们说这些评估是衡量产品如何运作的重要指标？
And it's when you start seeing that and you start seeing performance on evals going up, you start saying, "Okay, I think we have a product here."
当你开始看到这一点并且你开始看到评估性能上升时，你开始说，“好吧，我想我们这里有一个产品。”
Lenny Rachitsky: You made a kind of a comment along these same lines around evals that AI is almost capped in how amazing it can be by how good we are at evals.
Lenny Rachitsky：您围绕评估做了类似的评论，人工智能的惊人程度几乎取决于我们在评估方面的表现。
Does that resonate?
这引起共鸣吗？
Any more thoughts along those lines?
沿着这些思路还有更多的想法吗？
Kevin Weil: I mean, these models are their intelligences and intelligence is so fundamentally multidimensional so you can talk about a model being amazing at competitive coding, which may not be the same as that model being great at front-end coding-
Kevin Weil：我的意思是，这些模型就是它们的智能，而智能从根本上来说是多维的，所以你可以谈论一个在竞争性编码方面表现出色的模型，这可能与在前端编码方面表现出色的模型不同——
Kevin Weil: ...
凯文·威尔：...
may not be the same as that model being great at front-end coding or back-end coding or taking a whole bunch of code that's written in COBOL and turning it into Python.
可能与擅长前端编码或后端编码或采用 COBOL 编写的一大堆代码并将其转换为 Python 的模型不同。
And that's just within the software engineering world.
这还只是软件工程领域的事情。
So I think there's a sense in which you can think of these models as incredibly smart, very factually aware intelligences, but still most of the world's data, knowledge, process is not public.
所以我认为，从某种意义上说，你可以将这些模型视为非常聪明、非常了解事实的智能，但世界上大部分数据、知识和流程仍然不是公开的。
It's behind the walls of companies or governments or other things.
它位于公司、政府或其他事物的墙后。
And same way, if you were going to join a company, you would spend your first two weeks onboarding.
同样，如果你要加入一家公司，你会在前两周度过入职时间。
You'd be learning the company-specific processes.
您将学习公司特定的流程。
You'd get access to company-specific data.
您可以访问公司特定的数据。
The models are smart enough, you can teach them anything, but they need to have the raw data to learn from.
这些模型足够聪明，你可以教它们任何东西，但它们需要有原始数据来学习。
Kevin Weil: So there's a sense in which I think the future is really going to be incredibly smart, broad-based models that are fine-tuned and tailored with company-specific or use case-specific data so that they perform really well on company-specific, or use case-specific things.
Kevin Weil：因此，我认为未来确实会出现非常智能、基础广泛的模型，这些模型会根据公司特定或用例特定的数据进行微调和定制，以便它们在公司特定或用例特定的事情上表现得非常好。
And you're going to measure that with custom evals.
您将使用自定义评估来衡量这一点。
So what I was referring to is just like these models are really smart, you need to still teach them things if the data's not in their training set, and there's a huge amount of use cases that are not going to be in their training set because they're relevant to one industry or one company.
所以我指的是就像这些模型非常聪明一样，如果数据不在他们的训练集中，你仍然需要教他们一些东西，并且有大量的用例不会在他们的训练集中，因为它们与一个行业或一家公司相关。
Lenny Rachitsky: I'm just going to keep following the thread that you're leading us down, but I'm going to come back because I have more questions around some of these things.
Lenny Rachitsky：我将继续关注您引导我们的线索，但我会回来，因为我对其中一些事情还有更多问题。
So you came to a space that I think a lot of AI founders are thinking about is just, where's OpenAI not going to come squash me in the future?
所以你来到了一个我认为很多人工智能创始人都在思考的领域：OpenAI 未来在哪里不会压垮我？
Or one of the other foundational models.
或者其他基础模型之一。
So it's unclear to a lot of people just like, "Should I build a startup in this space or not?" Is there any advice you have or any guidance for where you think OpenAI, or just foundational models in general likely won't go and where you have an opportunity to build a company?
因此，很多人都不清楚，“我是否应该在这个领域建立一家初创公司？”对于您认为 OpenAI 或一般基础模型可能不会发展的领域以及您有机会建立公司的领域，您有什么建议或指导吗？
Kevin Weil: So this is something that Ev Williams used to say back at Twitter that's always stuck with me, which is, "No matter how big your company gets, no matter how incredible the people are, there are way more smart people outside your walls than there are inside your walls." And that's why we are so focused on building a great API.
凯文·威尔：埃夫·威廉姆斯在推特上说过的话一直让我印象深刻，那就是：“无论你的公司有多大，无论员工有多么不可思议，你的墙外的聪明人比你的墙内的聪明人多得多。”这就是为什么我们如此专注于构建出色的 API。
We have 3 million developers using our API.
我们有 300 万开发者使用我们的 API。
No matter how ambitious we are, how big we grow, by the way, we don't want to grow super big, there are so many use cases, places in the world where AI can fundamentally make our lives better.
不管我们有多么雄心勃勃，我们发展到多大，顺便说一句，我们不想变得超级大，世界上有很多用例和地方，人工智能可以从根本上让我们的生活变得更好。
We're not going to have the people.
我们不会有这些人。
We're not going to have the know-how to build most of these things.
我们不会拥有建造其中大部分东西的专业知识。
Kevin Weil: And I think, like I was saying, the data is industry-specific, use case-specific, behind certain company walls, things like that.
凯文·韦尔：我认为，就像我说的那样，数据是针对特定行业、特定用例的，在某些公司的墙后，诸如此类。
And there are immense opportunities in every industry and every vertical in the world to go build AI-based products that improve upon the state of the art.
世界上每个行业和每个垂直领域都存在巨大的机会来构建基于人工智能的产品，以改进最先进的技术。
And there's just no way we could ever do that ourselves.
我们自己根本不可能做到这一点。
We don't want to.
我们不想。
We if we did want to, and we're really excited to power that for 3 million-plus developers and way more in the future.
如果我们确实愿意的话，我们真的很高兴能够为超过 300 万开发者以及未来更多的开发者提供支持。
Lenny Rachitsky: Coming back to your earlier point about the tech changing constantly and getting faster, not exactly knowing what you'll have by the time you launch something in terms of the power, the model.
Lenny Rachitsky：回到你之前的观点，技术不断变化并且变得越来越快，但并不完全知道当你推出某种产品时，你会拥有什么功能、型号。
I'm curious what allows you to ship quickly and consistently in such great stuff?
我很好奇是什么让你们能够快速、一致地交付如此出色的产品？
And it sounds like one answer is bottoms-up empowered teams versus a very top-down roadmap that's planned out for a quarter.
听起来一个答案是自下而上的授权团队，而不是计划在一个季度内制定的自上而下的路线图。
What are some of those things that allow you to ship such great stuff so often, so quickly?
哪些因素可以让您如此频繁、如此快速地交付如此优质的产品？
Kevin Weil: Yeah.
凯文·威尔：是的。
I mean, we try and have a sense of where we're trying to go, point ourselves in a direction so that we have some rough sense of alignment.
我的意思是，我们尝试了解我们要走向何方，为自己指明方向，以便我们有一些粗略的一致性感。
Thematically, I don't for second, and we do quarterly roadmapping.
从主题上讲，我不介意，我们每季度都会制定路线图。
We laid out a year-long strategy.
我们制定了为期一年的战略。
I don't for a second believe that what we write down in these documents is what we're going to actually ship three months from now, let alone six or nine.
我一点也不相信我们在这些文件中写下的内容是我们从现在起三个月后实际交付的内容，更不用说六个月或九个月了。
But that's okay.
但没关系。
I think it's like an Eisenhower quote, "Plans are useless.
我认为这就像艾森豪威尔的一句话：“计划是没有用的。
Planning is helpful," which I totally subscribe to, especially in this world.
计划是有帮助的，”我完全同意这一点，尤其是在这个世界上。
It's really valuable.
真的很有价值。
If you think about quarterly road roadmapping for example, it's really valuable to have a moment where you stop and go, "Okay.
例如，如果您考虑季度路线图，那么有一个停下来走走的时刻真的很有价值，“好吧。
What did we do?
我们做了什么？
What worked?
什么有效？
What went well?
什么进展顺利？
What didn't go well?
什么事情进展不顺利？
What did we learn and now what do we think we're going to do next?"
我们学到了什么，现在我们认为下一步要做什么？”
Kevin Weil: And by the way, everybody has some dependencies.
凯文·韦尔：顺便说一句，每个人都有一些依赖性。
You need the infrastructure team to do the following things, partnership with research here.
您需要基础设施团队完成以下工作，并与此处的研究合作。
So you want to have a second to check your dependencies, make sure you're good to go and then start executing.
因此，您需要花一点时间检查您的依赖关系，确保一切顺利，然后开始执行。
We try and keep that really lightweight because it's not going to be right.
我们努力保持它的轻量级，因为它不会是正确的。
We're going to throw it out halfway because we will have learned new things.
我们会半途而废，因为我们会学到新东西。
So the moment of planning is helpful even if it's only partially.
因此，即使只是部分计划，此刻的计划也是有帮助的。
Kevin Weil: So I think just expecting that you're going to be super agile and that there's no sense writing a three month roadmap, let alone a year long roadmap because the technology's changing underneath you so quickly.
Kevin Weil：所以我认为仅仅期望你会变得超级敏捷，并且编写三个月的路线图是没有意义的，更不用说一年的路线图了，因为技术在你的脚下变化得如此之快。
We really do try and go very strongly bottoms up, subject to our overall directional alignment.
我们确实尝试非常强烈地自下而上，这取决于我们的整体方向调整。
We have great people.
我们有很棒的人。
We have engineers and PMs and designers and researchers who are passionate about the products they're building and have strong opinions about them and are also the ones building them.
我们有工程师、产品经理、设计师和研究人员，他们对他们正在构建的产品充满热情，并对这些产品有强烈的意见，而且他们也是这些产品的构建者。
So they have a real sense of what the capabilities are too, which is super important.
因此，他们也真正了解自己的能力，这一点非常重要。
Kevin Weil: So I think you want to be more bottoms up in this way.
凯文·韦尔：所以我认为你希望以这种方式更加自下而上。
So we operate that way.
所以我们就是这样运作的。
We are happy making mistakes.
我们很高兴犯错误。
We make mistakes all the time.
我们总是犯错误。
It's one of the things I really appreciate about Sam.
这是我真正欣赏萨姆的事情之一。
He pushes us really hard to move fast, but he also understands that with moving fast comes, we didn't quite get this right or that we launched this thing, it didn't work.
他非常努力地推动我们快速行动，但他也明白，随着快速行动的到来，我们并没有完全正确地做到这一点，或者我们推出了这个东西，它没有发挥作用。
We'll roll it back.
我们会将其回滚。
Look at our naming.
看看我们的命名。
Our naming is horrible.
我们的命名很糟糕。
Lenny Rachitsky: That was a lot of questions people had for you.
Lenny Rachitsky：人们向你提出了很多问题。
Model names, yeah.
型号名称，是的。
Kevin Weil: It's absolutely atrocious and we know it, and we will get around to fixing it at some point, but it's not the most important thing and so we don't spend a lot of time on it.
Kevin Weil：这绝对是残暴的，我们知道这一点，我们会在某个时候解决它，但这不是最重要的事情，所以我们不会花很多时间在上面。
Lenny Rachitsky: But it also shows you how it doesn't matter.
Lenny Rachitsky：但这也告诉你这并不重要。
Again, ChatGPT the most popular, fastest growing product in history, it's the number one AI, API and model.
再说一遍，ChatGPT 是历史上最受欢迎、增长最快的产品，它是排名第一的 AI、API 和模型。
So clearly it doesn't matter that much.
很明显，这并不重要。
Kevin Weil: And we name things like o3 mini high.
Kevin Weil：我们将诸如 o3 mini high 之类的东西命名为 o3 mini high。
Lenny Rachitsky: Man, I love it.
莱尼·拉奇茨基：伙计，我喜欢它。
Okay.
好的。
So you talked about roadmapping and bottoms up and I'm really curious, is there a cadence or a ritual of aligning with you or Sam or you review everything that's going out?
所以你谈到了路线图和自下而上的问题，我真的很好奇，是否有与你或 Sam 保持一致的节奏或仪式，或者你回顾所有正在发生的事情？
Is there a meeting every week or every month where you guys see what's happening?
你们每周或每月都有一次会议来了解正在发生的事情吗？
Kevin Weil: On key projects.
凯文·威尔：关于重点项目。
So we do product reviews and things like that, like you would expect.
所以我们会进行产品评论和类似的事情，就像您所期望的那样。
There isn't a ritual because there isn't...
没有仪式，因为没有……
I would never want us to be blocked on launching something, waiting for a review with me or Sam, if we can't get there.
如果我们无法到达目的地，我永远不希望我们在启动某些东西时受到阻碍，等待我或萨姆的审查。
If I'm traveling or Sam's busy or whatever, that's a bad reason for us not to ship.
如果我在旅行或者山姆很忙或者其他什么原因，这就是我们不发货的糟糕理由。
So obviously for the biggest, most high priority stuff, we have a pretty close beat on it, but we really try not to, frankly.
显然，对于最大、最优先的事情，我们已经非常接近了，但坦率地说，我们真的尽量不这样做。
We want to empower teams to move quickly, and I think it's more important to ship and iterate.
我们希望让团队能够快速行动，我认为交付和迭代更为重要。
Kevin Weil: So we have this philosophy, we call iterative deployment, and the idea is we're all learning about these models together.
Kevin Weil：所以我们有这样的理念，我们称之为迭代部署，这个想法是我们一起学习这些模型。
So there's a real sense in which it's way better to ship something even when you don't know the full set of capabilities and iterate together in public.
因此，即使您不知道完整的功能集并在公共场合一起迭代，交付某些东西也确实会更好。
And we co-evolve together with the rest of society as we learn about these things and where they're different and where they're good and bad and weird.
当我们了解这些事物以及它们的不同之处以及它们的好坏和奇怪之处时，我们与社会其他人共同进化。
I really like that philosophy.
我真的很喜欢这种哲学。
Kevin Weil: I think the other thing that ends up being a part of our product philosophy is the sense of model maximalism.
Kevin Weil：我认为最终成为我们产品哲学一部分的另一件事是模型极简主义的感觉。
The models are not perfect.
这些模型并不完美。
They're going to make mistakes.
他们会犯错误。
You could spend a lot of time building all kinds of different scaffolding around them.
您可能会花费大量时间围绕它们构建各种不同的脚手架。
And by the way, sometimes we do because sometimes there are kinds of errors that you just don't want to make, but we don't spend that much time building scaffolding around the parts that don't match that because our general mindset is in two months there's going to be a better model and it's going to blow away whatever the current set of limitations are.
顺便说一句，有时我们会这样做，因为有时会出现一些你不想犯的错误，但我们不会花太多时间围绕不匹配的部分构建脚手架，因为我们的总体思维是在两个月内将会出现一个更好的模型，并且它将消除当前的限制。
Kevin Weil: So if you're building, and we say this to developers too, if you're building and the product that you're building is right on the edge of the capabilities of the models, keep going, because you're doing something right because you give it another couple months and the models are going to be great, and suddenly the product that you have that just barely worked is really going to sing.
Kevin Weil：所以，如果你正在构建，我们也对开发人员这么说，如果你正在构建并且你正在构建的产品正好处于模型功能的边缘，请继续前进，因为你正在做正确的事情，因为你再给它几个月，模型将会很棒，突然之间，你刚刚几乎无法工作的产品真的会唱歌。
And that's how you make sure that you're really pushing the envelope and building new things.
这就是你如何确保自己真正突破极限并创造新事物的方法。
Lenny Rachitsky: I had the founder of Bolt on the podcast, StackBlitz is the company name, and he shared this story that they've been working on this product for seven years behind the scenes and it was failing.
Lenny Rachitsky：我在播客上看到了 Bolt 的创始人，StackBlitz 是公司名称，他分享了这个故事，他们在幕后开发这个产品七年了，但失败了。
Nothing was happening.
什么也没发生。
And then all of a sudden it was, sorry to mention a competitor, but Claude came out or a Sonnet 3.5 came out and all of a sudden everything worked and they've been building all this time and finally it worked.
然后突然之间，很抱歉提到竞争对手，但是 Claude 出来了，或者 Sonnet 3.5 出来了，突然间一切都正常了，他们一直在构建，最后终于成功了。
And I hear that a lot with YC, just like things that never were possible now are just becoming possible every few months with the updates to the models.
我在 YC 上听到了很多这样的话，就像以前不可能的事情现在随着模型的更新每隔几个月就变得可能一样。
Lenny Rachitsky: Let me actually ask this, I wasn't planning to ask this, but I'm curious if you have any quick thoughts just why is Sonnet so good at coding, and thoughts on your stuff getting as good and better at actual coding?
Lenny Rachitsky：让我实际上问这个问题，我本来不打算问这个，但我很好奇您是否有任何快速的想法，为什么 Sonnet 如此擅长编码，以及您的想法在实际编码方面变得越来越好？
Kevin Weil: Yeah.
凯文·威尔：是的。
I mean, kudos to Anthropic.
我的意思是，向人类致敬。
They've built very good coding models.
他们建立了非常好的编码模型。
No doubt.
无疑。
We think that we can do the same.
我们认为我们也可以这样做。
Maybe by the time this podcast has shipped, we'll have more to say, but either way, all credit to them.
也许当这个播客发布时，我们会有更多的话要说，但无论如何，这一切都归功于他们。
I think intelligence is really multi-dimensional and so I think the model providers...
我认为情报确实是多维的，所以我认为模型提供者......
It used to be that OpenAI had this massive model lead, 12 months or something ahead of everybody else.
过去，OpenAI 拥有巨大的模型领先优势，领先其他所有人 12 个月左右。
That's not true anymore.
那不再是真的了。
I like to think we still have a lead.
我认为我们仍然领先。
I'd argue that we do, but it's certainly not a massive one.
我认为我们确实这样做了，但这肯定不是一个大规模的。
And that means that there are going to be different places where the Google models are really good or where Anthropic models are really good, or where we're really good and our competitors are like, "We got to get better at that." And it actually is easier to get better at a certain thing once someone's proved it possible than it is to forge a path through the jungle and doing something brand new.
这意味着在不同的地方，谷歌模型真的很好，或者人择模型真的很好，或者我们真的很好，而我们的竞争对手会说，“我们必须在这方面做得更好。”事实上，一旦有人证明了某件事是可能的，在某件事上做得更好比在丛林中开辟一条道路并做一些全新的事情更容易。
Kevin Weil: So I just think as an example, it was like nobody could break 4 minutes in the mile, and then finally somebody did and the next year 12 more people did it.
Kevin Weil：所以我想举个例子，就像没有人能在一英里比赛中打破 4 分钟，然后终于有人做到了，第二年又有 12 个人做到了。
I think there's that all over the place and it just means that competition is really intense, and consumers are going to win and developers are going to win and businesses are going to win in a big way from that.
我认为这种情况无处不在，这意味着竞争非常激烈，消费者将获胜，开发商将获胜，企业将从中获得很大的胜利。
It's part of why the industry moves so fast, but all respect to the other big model providers.
这是该行业发展如此之快的部分原因，但也要尊重其他大型模型提供商。
Models are getting really good.
模型变得非常好。
We're going to move as fast as we can and I think we've got some good stuff coming.
我们将尽快采取行动，我认为我们已经取得了一些好成绩。
Lenny Rachitsky: Exciting.
莱尼·拉奇茨基：令人兴奋。
This makes me also think about, in many ways other models are better at certain things, but somehow ChatGPT is the...
这也让我思考，在很多方面其他模型在某些事情上做得更好，但不知何故 ChatGPT 是......
If you look at all the awareness numbers and usage numbers, it's like no matter where you guys are in the rankings, people seem to just think of AI ChatGPT almost as the same.
如果你看一下所有的认知度数据和使用量数据，就会发现无论你们在排名中处于什么位置，人们似乎只认为 AI ChatGPT 几乎是一样的。
What do you think you did right to win in the consumer mindset, at least at this point and awareness in the world?
您认为您做了哪些正确的事情来赢得消费者的心态（至少在这一点上）以及世界的认知度？
Kevin Weil: I think being first helps, which is one of the reasons why we're so focused on moving quickly.
凯文·韦尔：我认为先行会有所帮助，这也是我们如此专注于快速行动的原因之一。
We like being the first to launch new capabilities.
我们喜欢成为第一个推出新功能的人。
Things like deep research.
诸如深入研究之类的事情。
Our models, they can do a lot of things.
我们的模型，他们可以做很多事情。
So they can take real-time video input, you have speech to speech, you can do speech to text and text to speech.
所以他们可以进行实时视频输入，你可以进行语音到语音，你可以进行语音到文本和文本到语音。
They can do deep research.
他们可以进行深入的研究。
They can operate on a canvas, they can write code.
他们可以在画布上操作，可以编写代码。
So ChatGPT can be this one- stop-shop where all the things that you want to do are possible.
因此，ChatGPT 可以成为一站式商店，您想做的所有事情都可以实现。
And as we go forward in it, we have more agentic tools like Operator where it's browsing for you and doing things for you on the web, more and more you're going to be able to come to this one place to ChatGPT, give it instructions and have it accomplish real things for you in the world.
随着我们的前进，我们有了更多的代理工具，比如 Operator，它可以在网络上为你浏览并为你做事，越来越多的你将能够来到 ChatGPT 这个地方，给它指令，让它为你在世界上完成真正的事情。
There's something fundamentally valuable in that.
这里面有一些根本上有价值的东西。
So we think a lot about that.
所以我们对此思考了很多。
We try to move really fast so that we are always the most useful place for people to come to.
我们努力快速行动，以便我们始终成为人们最有用的地方。
Lenny Rachitsky: What would you say is the most counterintuitive thing that you've learned after building AI products or working at OpenAI, something that's just like, "I did not expect that?"
Lenny Rachitsky：您认为在构建 AI 产品或在 OpenAI 工作后学到的最违反直觉的事情是什么，就像“我没想到会这样？”
Kevin Weil: I don't know, maybe I should have expected this, but one of the things that's been funny for me is the extent to which you're trying to figure out how some product should work with AI, or even why some AI thing happens to be true, you can often reason about it the way you would reason about another human and it works.
Kevin Weil：我不知道，也许我应该预料到这一点，但对我来说有趣的一件事是，你试图弄清楚某些产品应该如何与人工智能一起工作，或者甚至为什么某些人工智能的事情碰巧是真的，你经常可以像推理另一个人一样推理它，它是有效的。
So maybe a couple examples.
也许举几个例子。
When we were first launching our reasoning model, we were the first to build a model that could reason, that could, instead of giving you just a quick system one answer right away to every question you asked, it was the third Emperor of the Holy Roman Empire, here's an answer.
当我们第一次启动我们的推理模型时，我们是第一个构建一个可以推理的模型，它可以，而不是立即给你一个快速的系统一答案，立即回答你提出的每个问题，这是神圣罗马帝国的第三位皇帝，这是一个答案。
Kevin Weil: You could ask it hard questions and it would reason.
凯文·威尔：你可以问它一些棘手的问题，它会推理。
The same way that if I asked you to do a crossword puzzle, you couldn't just snap fill in everything.
就像我让你做填字游戏一样，你不能简单地填写所有内容。
You would be, "Well, okay.
你会说：“好吧，好吧。
On this one across, I think it could be one of these two, but that means there's an A here.
在这一横线上，我认为它可能是这两个之一，但这意味着这里有一个 A。
So that one has to be this, away, back track, step-by-step build up from where you are." Same way you answer any difficult logistical problem, any scientific problem.
因此，必须是这样的，离开，原路返回，从你所在的地方一步一步地建立起来。”就像你回答任何困难的后勤问题、任何科学问题一样。
So this reasoning breakthrough was big, but it was also the first time that a model needed to sit and think.
所以这个推理突破是很大的，但这也是模型第一次需要坐下来思考。
And that's a weird paradigm for a consumer product.
对于消费品来说，这是一个奇怪的范例。
You don't normally have something where you might need to hang out for 25 seconds after you ask a question.
通常情况下，您不会在提问后需要等待 25 秒。
Kevin Weil: So we were trying to figure out what's the UI for this?
Kevin Weil：所以我们试图弄清楚这个的 UI 是什么？
With deep research where the model's going to go and think for 25 minutes sometimes, it's actually not that hard because you're not going to sit and watch it for 25 minutes.
通过深入研究，模型有时会思考 25 分钟，实际上这并不难，因为你不会坐下来观看它 25 分钟。
You're going to go do something else.
你要去干别的事了。
You're going to go to another tab or go get lunch or whatever, and then you'll come back and it's done when it's like 20, 25 seconds or 10 seconds, it's a long time to wait, but it's not long enough to go to do something else.
你要去另一个选项卡或者去吃午饭什么的，然后你会回来，大约 20、25 秒或 10 秒就完成了，等待的时间很长，但还不足以去做其他事情。
Kevin Weil: So you can think, if you asked me something that I needed to think for 20 seconds to answer, what would I do?
凯文·威尔：所以你可以想一想，如果你问我一些我需要思考 20 秒才能回答的问题，我会怎么做？
I wouldn't just go mute and not say anything and shut down for 20 seconds and then come back.
我不会只是保持沉默，什么也不说，然后关机 20 秒，然后再回来。
So we shouldn't do that.
所以我们不应该这样做。
We shouldn't just have a slider sitting there.
我们不应该只放一个滑块。
That's annoying.
这很烦人。
But I also wouldn't just start babbling every single thought that I had.
但我也不会开始胡言乱语我的每一个想法。
So we probably shouldn't just expose the whole chain of thought as the model's thinking, but I might go like, "That's a good question.
因此，我们可能不应该仅仅将整个思想链暴露为模型的思维，但我可能会说，“这是一个好问题。
All right." I might approach it like that and then think.
好吧。”我可能会这样处理然后思考。
You're maybe giving little updates and that's actually what we ended up shipping.
您可能只提供了一些更新，而这实际上就是我们最终发布的内容。
Kevin Weil: You have similar things where you can find situations where you get better thinking sometimes out of a group of models that all try and attack the same problem, and then you have a model that's looking at all their outputs and integrating it and then giving you a single answer at the end.
Kevin Weil：你也有类似的情况，你可以找到这样的情况，有时你可以从一组模型中得到更好的思考，这些模型都试图解决相同的问题，然后你有一个模型，它会查看它们的所有输出并将其集成，然后在最后给你一个单一的答案。
I mean, sounds a little bit like brainstorming.
我的意思是，听起来有点像头脑风暴。
I certainly have better ideas when I get in a room and brainstorm with other people because they think differently than me.
当我走进一个房间并与其他人进行头脑风暴时，我当然有更好的想法，因为他们的想法与我不同。
So anyways, there's just all these situations where you can actually reason about it like a group of humans or an individual human and it works, which I don't know, maybe I shouldn't have been surprised but I was.
所以无论如何，在所有这些情况下，你实际上可以像一群人或个人一样对其进行推理，并且它有效，我不知道，也许我不应该感到惊讶，但我确实感到惊讶。
Lenny Rachitsky: That is so interesting because when I see these models operate, I never even thought about you guys designing that experience.
Lenny Rachitsky：这太有趣了，因为当我看到这些模型运行时，我什至从未想过你们设计了这种体验。
To me, it just feels like this is what the LLM does.
对我来说，感觉这就是法学硕士所做的事情。
It just sits there and tells me what it's thinking.
它只是坐在那里告诉我它在想什么。
And I love this point you're making of let's make it feel like a human operating and well, how does a human operate?
我喜欢你提出的这一点，让我们感觉像是一个人在操作，那么，人是如何操作的呢？
Well, they just talk aloud.
好吧，他们只是大声说话。
They think, here's the thing I should explore.
他们认为，这就是我应该探索的事情。
And I love that deep sequence to the extreme of that where they're just like, "Here's everything I'm doing and thinking." And people actually like that too, I guess.
我喜欢那种深刻的序列，到了极致，他们就像是，“这就是我正在做和想的一切。”我想人们实际上也喜欢这样。
Was that surprising to you, "Maybe that could work too.
你是否感到惊讶，“也许这也可行。
People seem to like everything?"
人们似乎什么都喜欢？”
Kevin Weil: Yeah.
凯文·威尔：是的。
We learned from that actually because when we first launched it, we gave you the subheadings of what the model was thinking about, but not much more.
我们从中学到了很多东西，因为当我们第一次推出它时，我们给了你模型所考虑的副标题，但仅此而已。
And then deep seek launched and it was a lot and we went, I don't know if everyone wants that.
然后深度搜索启动了，数量很多，我们去了，我不知道是否每个人都想要这样。
There's some novelty effect to seeing what the model's really thinking about.
了解模型真正的想法有一些新颖的效果。
We felt that too when we were looking at it internally.
我们在内部审视的时候也有这样的感觉。
It's interesting to see the model's chain of thought, but it's not...
看到模型的思想链很有趣，但事实并非如此......
I think at the scale of 400 million people, you don't want to see the model babble a bunch of things.
我认为在4亿人的规模下，你不想看到模型胡言乱语。
Kevin Weil: So what we ended up doing was summarizing it in interesting ways.
凯文·韦尔：所以我们最终所做的是以有趣的方式进行总结。
So instead of just getting the subheadings, you're getting one or two sentences about how it's thinking about it and you can learn from that.
因此，您不仅可以获得副标题，还可以获得一两句话，了解它是如何思考的，您可以从中学习。
So we tried to find a middle ground that we thought was an experience would be meaningful for most people, but showing everybody three paragraphs is probably not the right answer.
因此，我们试图找到一个中间立场，我们认为这种体验对大多数人来说都是有意义的，但向每个人展示三段可能不是正确的答案。
Lenny Rachitsky: This reminds me of something else you said at the summit that has really stuck with me, this idea that chat, people always make fun of chat is not the future interface for how we interact with AI, but you made this really interesting point that may argue the other side, which is, as humans we interface by talking and the IQ of a human can span from really low to really high and it all works talking to them and chat is the same thing and it can work on all kinds of intelligence levels.
Lenny Rachitsky：这让我想起你在峰会上说过的另一件事，让我印象深刻，即聊天，人们总是取笑聊天，这并不是我们与人工智能互动的未来界面，但你提出了一个非常有趣的观点，可能会与另一方争论，即，作为人类，我们通过交谈进行交流，人类的智商可以从非常低到非常高，与他们交谈和聊天是一样的，它可以在各种智力水平上发挥作用。
Maybe I just shared it, but I guess anything there about just why chat actually ends up being such an interesting interface for LLMs?
也许我只是分享了它，但我猜想为什么聊天实际上最终成为法学硕士如此有趣的界面？
Kevin Weil: Yeah.
凯文·威尔：是的。
I don't know, maybe this is one of those things I believe that most people don't believe, but I actually think chat is an amazing interface because it's so versatile.
我不知道，也许这是我相信大多数人不相信的事情之一，但我实际上认为聊天是一个令人惊奇的界面，因为它是如此多才多艺。
People tend to go, "Chat.
人们倾向于“聊天”。
Yeah.
是的。
We'll figure out something better." And I think it's incredibly universal because it is the way we talk.
我们会想出更好的办法。”我认为这是非常普遍的，因为这就是我们说话的方式。
I can talk to you verbally like we're talking now.
我可以像我们现在说话一样和你口头交谈。
We can see each other and interact.
我们可以看到对方并互动。
We can talk on WhatsApp and be texting each other, but all of these things is this unstructured method of communication and that's how we operate.
我们可以在 WhatsApp 上聊天，也可以互相发短信，但所有这些都是非结构化的沟通方式，这就是我们的运作方式。
Kevin Weil: If I had some more rigid interface that I was allowed to use when we spoke, I would be able to speak to you about far fewer things and it would actually get in the way of us having maximum communication bandwidth.
凯文·威尔：如果我有一些更严格的界面，允许我在我们交谈时使用，我就能和你谈论的事情少得多，而且它实际上会妨碍我们拥有最大的通信带宽。
So there's something magical.
所以有一些神奇的东西。
And by the way, in the past it never worked because there wasn't a model that was good at understanding all of the complexity and nuances of human speech, and that's the magic of LLMs.
顺便说一句，在过去它从来没有起作用，因为没有一个模型能够很好地理解人类语音的所有复杂性和细微差别，而这就是法学硕士的魔力。
So to me, it's like an interface that's exactly fit to the power of these things.
所以对我来说，它就像一个完全适合这些东西的功能的界面。
And that doesn't mean that it always has to be just like I don't necessarily always want to type, but you do want that very open-ended, flexible communication medium, it may be that we're speaking and the model's speaking back to me, but you still want the very lowest common denominator, no restrictions way of interacting.
这并不意味着它总是必须像我不一定总是想打字一样，但你确实想要那种非常开放、灵活的沟通媒介，可能是我们在说话，模型也在对我说话，但你仍然想要最低的共同点，没有限制的互动方式。
Lenny Rachitsky: That is so interesting.
Lenny Rachitsky：这太有趣了。
That's really changed the way I think about this stuff is that point that chat is just so good for this very specific problem of talking to superintelligence basically.
这确实改变了我对这个问题的看法，因为聊天对于与超级智能对话这一非常具体的问题来说非常有用。
Kevin Weil: By the way, I think it's not that it's only chat either.
Kevin Weil：顺便说一句，我认为这也不仅仅是聊天。
If you have high volume use cases where they're more prescribed and you don't actually need the full generality, there are many use cases where it's better to have something that's less flexible, more prescribed, faster to specific task, and those are great too, and you can build all sorts of those.
如果你有大量的用例，它们被更多地规定，并且你实际上并不需要完全的通用性，那么在许多用例中，最好有一些不太灵活、更规定、更快地完成特定任务的东西，这些也很棒，你可以构建各种各样的用例。
But you still want chat as this baseline for anything that falls out of whatever vertical you happen to be building for.
但您仍然希望将聊天作为任何超出您正在构建的垂直领域的基线。
It's like a catch-all for every possible thing you'd ever want to express to a model.
它就像一个包罗万象的东西，包含了你想要向模型表达的所有可能的东西。
Lenny Rachitsky: I'm excited to chat with Christina Gilbert, the founder of OneSchema, one of our long-time podcast sponsors.
Lenny Rachitsky：我很高兴能与 OneSchema 的创始人 Christina Gilbert 聊天，OneSchema 是我们的长期播客赞助商之一。
Hi, Christina.
嗨，克里斯蒂娜。
Christina Gilbert: Yes.
克里斯蒂娜·吉尔伯特：是的。
Thank you for having me on, Lenny.
谢谢你邀请我参加，莱尼。
Lenny Rachitsky: What is the latest with OneSchema?
Lenny Rachitsky：OneSchema 的最新动态是什么？
I know you now with some of my favorite companies like Ramp, Vanta, Scale and Watershed.
我现在认识了您和我最喜欢的一些公司，例如 Ramp、Vanta、Scale 和 Watershed。
I heard that you just launched a new product to help product teams import CSVs from especially tricky systems like ERPs?
我听说你们刚刚推出了一款新产品来帮助产品团队从 ERP 等特别棘手的系统导入 CSV？
Christina Gilbert: Yes.
克里斯蒂娜·吉尔伯特：是的。
So we just launched OneSchema FileFeeds, which allows you to build an integration with any system in 15 minutes as long as you can export a CSV to an SFTP folder.
因此，我们刚刚推出了 OneSchema FileFeeds，只要您可以将 CSV 导出到 SFTP 文件夹，您就可以在 15 分钟内与任何系统构建集成。
We see our customers all the time getting stuck with hacks and workarounds, and the product teams that we work with don't have to turn down prospects because their systems are too hard to integrate with.
我们经常看到我们的客户陷入黑客攻击和解决方法的困境，而与我们合作的产品团队不必因为他们的系统太难集成而拒绝潜在客户。
We allow our customers to offer thousands of integrations without involving their engineering team at all.
我们允许客户提供数千种集成，而无需他们的工程团队参与。
Lenny Rachitsky: I can tell you that if my team had to build integrations like this, how nice would it be to be able to take this off my roadmap and instead, use something like OneSchema and not just to build it, but also to maintain it forever.
Lenny Rachitsky：我可以告诉你，如果我的团队必须构建这样的集成，那么如果能够将其从我的路线图上删除，并使用像 OneSchema 这样的东西，并且不仅可以构建它，而且可以永远维护它，那该多好啊。
Christina Gilbert: Absolutely, Lenny.
克里斯蒂娜·吉尔伯特：当然，莱尼。
We've heard so many horror stories of multi-day outages from even just a handful of ad records.
我们已经从少量的广告记录中听到了许多多天停机的恐怖故事。
We are laser-focused on integration reliability to help teams end all of those distractions that come up with integrations.
我们高度关注集成可靠性，以帮助团队消除集成带来的所有干扰。
We have a built-in validation layer that stops any bad data from entering your system, and OneSchema will notify your team immediately of any data that looks incorrect.
我们有一个内置的验证层，可以阻止任何不良数据进入您的系统，并且 OneSchema 将立即通知您的团队任何看起来不正确的数据。
Lenny Rachitsky: I know that importing incorrect data can cause all kinds of pain for your customers, and quickly lose their trust.
Lenny Rachitsky：我知道导入不正确的数据会给您的客户带来各种痛苦，并很快失去他们的信任。
Christina, thank you for joining us.
克里斯蒂娜，感谢您加入我们。
And if you want to learn more, head on over to oneschema.co. That's oneschema.co.
如果您想了解更多信息，请访问 oneschema.co。那是 oneschema.co。
Lenny Rachitsky: I want to come back to that you talked about researchers and their relationship with product teams.
Lenny Rachitsky：我想回到您谈到的研究人员及其与产品团队的关系。
I imagine a lot of innovation comes from researchers just like having an inkling and then building something amazing and then releasing it, and some ideas come from PMs and engineers.
我想很多创新都来自研究人员，就像有一个想法，然后构建一些令人惊奇的东西，然后发布它，有些想法来自产品经理和工程师。
How do those teams collaborate?
这些团队如何协作？
Does every team have a PM?
每个团队都有 PM 吗？
Is it a lot of research-led stuff?
是不是有很多以研究为主导的东西？
Give us a sense of just where ideas and products come from mostly.
让我们了解创意和产品的主要来源。
Kevin Weil: It's an area where we're evolving a lot.
Kevin Weil：这是一个我们正在不断发展的领域。
I'm really excited about it, frankly.
坦白说，我对此感到非常兴奋。
I think if you go back a couple of years when ChatGPT was just getting started, obviously, I wasn't in OpenAI, but...
我想如果你回到几年前 ChatGPT 刚刚起步的时候，显然我并不在 OpenAI 中，但是……
Kevin Weil: Obviously I wasn't an Open AI, but...
Kevin Weil：显然我不是开放人工智能，但是......
We were more of a pure research company at the time.
当时我们更像是一家纯粹的研究公司。
Chat GPT, if you remember, was a low-key research preview.
如果你还记得的话，Chat GPT 是一个低调的研究预览。
Kevin Weil: Yeah.
凯文·威尔：是的。
It wasn't a thing that the team launched thinking it was going to be this massive product.
团队推出这款产品时并没有考虑到它会是一个大型产品。
Kevin Weil: And it was just a way that we were going to let people play with and iterate on the models.
Kevin Weil：这只是我们让人们玩弄和迭代模型的一种方式。
So we were primarily a research company, a world-class research company, and as ChatGPT has grown and as we've built our B-to-B products and our APIs and other things, now we're more of a product company than we were.
因此，我们主要是一家研究公司，一家世界级的研究公司，随着 ChatGPT 的发展以及我们构建了 B2B 产品、API 和其他产品，现在我们比以前更像是一家产品公司。
I still think we can't...
我仍然认为我们不能...
Open AI should never be a pure product company.
开放AI永远不应该是一家纯粹的产品公司。
We need to be both a world-class research company and a world-class product company, and the two need to really work together, and that's the thing that I think we've been getting much better at over the last six months.
我们需要成为世界一流的研究公司和世界一流的产品公司，两者需要真正合作，我认为这就是我们在过去六个月里做得更好的事情。
If you treat those things separately and the researchers go do amazing things and build models and then they get to some state and then the product and engineering teams go take them and do something with them, we're effectively just an API consumer of our own models.
如果你单独对待这些事情，研究人员去做一些令人惊奇的事情并构建模型，然后他们达到某种状态，然后产品和工程团队去接受它们并用它们做一些事情，那么我们实际上只是我们自己模型的 API 消费者。
Kevin Weil: The best products though are going to be, it's like I was talking about with deep research, it's a lot of iterative feedback.
Kevin Weil：最好的产品将会是，就像我所说的深入研究，这是很多迭代的反馈。
It's understanding the products you're trying to sell or the problems you're trying to solve, building evals for them, using those evals to go gather data and fine-tune models to get them to be better at these use cases that you're looking to solve.
它是了解您想要销售的产品或您想要解决的问题，为它们构建评估，使用这些评估来收集数据并微调模型，以使它们更好地处理您想要解决的这些用例。
It's a huge amount of back and forth to do it well.
要做好这件事需要大量的来回。
And I think the best products are going to be ENG product design and research working together as a single team to build novel things.
我认为最好的产品将是 ENG 产品设计和研究作为一个团队共同努力构建新颖的产品。
So that's actually how we're trying to operate with basically anything that we build.
这实际上就是我们尝试对我们构建的任何东西进行操作的方式。
It's a new muscle for us because we're kind of new as a product company, but it's one that people are really excited about because we've seen every time we do it, we build something awesome, and so now every product starts like that.
这对我们来说是一种新的力量，因为我们作为一家产品公司是新的，但人们真的很兴奋，因为我们每次做这件事时都看到，我们创造了一些很棒的东西，所以现在每个产品都是这样开始的。
Lenny Rachitsky: How many product managers do you have at Open AI?
Lenny Rachitsky：Open AI 有多少位产品经理？
I don't know if you share that number, but if you do.
我不知道你是否分享这个号码，但如果你分享的话。
Kevin Weil: Not that many, actually.
凯文·威尔：实际上，没有那么多。
I don't know, 25.
我不知道，25岁。
Maybe it's a little more than that.
也许还不止于此。
My personal belief is that you want to be pretty PM light as an organization just in general.
我个人的信念是，作为一个组织，总体而言，您希望成为一个相当轻松的 PM。
I say this with love because I am a PM, but too many PMs causes problems.
我满怀爱意地这么说，因为我是一名产品经理，但过多的产品经理会带来问题。
We'll fill the world with decks and ideas versus execution.
我们将让世界充满套牌和想法，而不是执行。
So I think it's a good thing when you have a PM that is working with maybe slightly too many engineers because it means they're not going to get in and micromanage.
因此，我认为当你的产品经理与过多的工程师一起工作时，这是一件好事，因为这意味着他们不会介入并进行微观管理。
You're going to leave a lot of influence and responsibility with the engineers to make decisions.
您将把很大的影响力和责任留给工程师来做决策。
It means you want to have really product-focused engineers, which we're fortunate to have.
这意味着您想要拥有真正以产品为中心的工程师，我们很幸运拥有这一点。
We have an amazingly product focused, high agency engineering team.
我们拥有一支以产品为中心的高代理工程团队。
But when you have something like that, you have a team that feels super empowered, you have a PM that's trying to really understand the problems and gently guide the team a little bit but has too much going on to get too far into the details, and you end up being able to move really fast.
但是当你有这样的事情时，你有一个感觉超级强大的团队，你有一个PM，他试图真正理解问题并温和地指导团队一点，但有太多的事情要做，无法深入细节，你最终能够快速行动。
So that's kind of the philosophy we take.
这就是我们所采取的理念。
Kevin Weil: We want Product ENG leads and product engineers all the way through.
Kevin Weil：我们希望产品 ENG 领导和产品工程师能够自始至终。
We want not too many PMs, but really awesome, high quality ones, and so far that seems to be working pretty well.
我们不需要太多的 PM，但需要真正出色、高质量的 PM，到目前为止，这似乎运作得很好。
Lenny Rachitsky: I imagine being a PM at Open AI is a dream come true for a lot of people.
Lenny Rachitsky：我想成为 Open AI 的产品经理对于很多人来说是梦想成真。
At the same time, I imagine it's not a fit for a lot of people.
同时，我认为它不适合很多人。
There's researchers involved, very product minded engineers.
有研究人员参与其中，他们是非常具有产品意识的工程师。
What do you look for in the PMs that you hire there for folks that are like, "Maybe I shouldn't go work there.
对于那些会说“也许我不应该去那里工作。
I shouldn't even think about that."
我什至不应该想这个。”
Kevin Weil: I think, I've said this a few times, but high agency is something that we really look for, people that are not going to come in and wait for everyone else to allow them to do something, they're just going to see a problem and go do it.
凯文·威尔：我想，我已经说过好几次了，但是我们真正寻找的是高代理权，人们不会进来等待其他人允许他们做某事，他们只是会看到问题并去做。
It's just a core part of how we work.
这只是我们工作方式的核心部分。
I think people that are happy with ambiguity, because there is a massive amount of ambiguity here, it is not the kind of place, and we have trouble sometimes with more junior PMs because of this, because it's just not the place where someone is going to come in and say, "Okay, here's the landscape, here's your area, I want you to go do this thing." And that's what you want as an early career PM.
我认为人们对模糊性感到满意，因为这里有大量的模糊性，这不是那种地方，因此我们有时会与更多的初级产品经理遇到麻烦，因为这不是一个有人会进来说“好吧，这里是风景，这是你的区域，我希望你去做这件事”的地方。这就是作为一名职业生涯早期的 PM 所想要的。
I mean, no one here has time and the problems are too ill-formed and we're figuring them all out as we go.
我的意思是，这里没有人有时间，而且问题的形式太不合理，我们正在边走边解决所有问题。
And so high agency, very comfortable with ambiguity, ready to come in and help execute and move really quickly.
如此高的代理权，对模糊性非常满意，准备好介入并帮助执行和快速行动。
That's kind of our recipe.
这就是我们的食谱。
Kevin Weil: And I think also happy leading through influence because...
凯文·韦尔：我认为通过影响力进行领导也很高兴，因为……
I mean it's usual as a PM, people don't report to you, your team doesn't report to you, et cetera, but you also have the complexity of a research function, which is even more sort of self-directed and it's really important to build a good rapport with the research team.
我的意思是，作为产品经理，人们不向你汇报，你的团队也不向你汇报，等等，但你也有研究职能的复杂性，这更加自我导向，与研究团队建立良好的融洽关系非常重要。
I think the EQ side of things is also super important for us.
我认为情商方面对我们来说也非常重要。
Lenny Rachitsky: I know at most companies, a PM comes in and they're just like, "Why do we need you?" And as a PM you have to earn trust and help people see the value, and I feel like at Open AI it's probably a very extreme version of that where they're like, "Why do we need this person?
Lenny Rachitsky：我知道在大多数公司，PM 进来时他们都会问：“我们为什么需要你？”作为一名产品经理，你必须赢得信任并帮助人们看到价值，我觉得在 Open AI，这可能是一个非常极端的版本，他们会说，“为什么我们需要这个人？
We have researchers, engineers, what are you going to do here?"
我们有研究人员、工程师，你们要来这里做什么？”
Kevin Weil: Yeah, I think people appreciate it done right, but you bring people along.
凯文·韦尔：是的，我认为人们会欣赏做得正确的做法，但你要带领人们一起行动。
I think one of the most important things a PM can do well is be decisive.
我认为，PM 能做好的最重要的事情之一就是果断。
So there's a real fine line.
所以有一条真正的细线。
You don't want to be making...
你不想让...
I mean it's kind of like, I don't love the PM as the CEO of the product illusion all the time, but just like Sam in his role would be making mistakes if he made every single decision in every meeting that he was in.
我的意思是，这有点像，我不喜欢 PM 作为产品幻想的首席执行官，但就像 Sam 一样，如果他在参加的每次会议中做出每一个决定，他就会犯错误。
And he would also be making mistakes if he made no decisions in any meetings that he was in, right?
如果他在参加的任何会议上都没有做出决定，他也会犯错误，对吧？
It's understanding when to defer to your team and to let people innovate.
它是理解何时服从你的团队并让人们创新。
And when there is a decision to be made that people either don't feel comfortable with or don't feel empowered to make, or a decision that has too many different disparate pros and cons that are spread out across a big group and someone needs to be decisive and make a call, it's a really important trait of a CEO.
当做出的决定让人们感到不舒服或感觉没有权力做出时，或者一个决定有太多不同的利弊分散在一个大群体中时，需要有人果断并做出决定，这就是首席执行官的一个非常重要的特质。
Kevin Weil: It's something Sam does well, and it's also a really important trait of a PM kind of at a more microscopic level.
Kevin Weil：这是 Sam 擅长的事情，而且从更微观的层面来说，这也是 PM 的一个非常重要的特质。
So because there's so much ambiguity, it's not obvious what the answer is in a lot of cases, and so having a PM that can come in and...
因为存在太多的歧义，所以在很多情况下答案并不明显，所以有一个 PM 可以介入并......
And by the way, this doesn't need to be a PM, I'm perfectly happy if it's anybody else, but I kind of look to the PM to say, if there's ambiguity and no one's making a call, you better make sure that we get a call made and we move forward.
顺便说一句，这不一定是 PM，如果是其他人，我会非常高兴，但我希望 PM 说，如果存在歧义并且没有人打电话，你最好确保我们接到电话，然后我们继续前进。
Lenny Rachitsky: This touches on a few posts I've done of just, where is AI going to take over work that we do versus help us with various work?
Lenny Rachitsky：这涉及到我刚刚发表的几篇文章，人工智能将在哪里接管我们所做的工作，而不是帮助我们完成各种工作？
So let me come at this question from a different direction of just how AI impacts product teams and hiring, things like that.
因此，让我从不同的方向来回答这个问题，即人工智能如何影响产品团队和招聘等。
So first of all, there's all this talk of LM's doing our coding for us, and 90% of code is going to be written by AI in a year.
首先，人们都在谈论 LM 为我们编码，而 90% 的代码将在一年内由 AI 编写。
Dario at Anthropic said that.
Anthropic 的达里奥这么说。
At the same time, you guys are all hiring engineers like crazy, PM's like crazy.
与此同时，你们都在疯狂地招聘工程师、PM。
Every function is dead, but you're still hiring every single one.
每个职能部门都已经死了，但你仍然在招聘每一个职能部门。
I guess just, first of all, let me just ask this, how do you and the team, say engineers, PMs, use AI in your work?
我想，首先，让我问一下，你和你的团队，比如工程师、产品经理，如何在工作中使用人工智能？
Is there anything that's really interesting or things that you think people are sleeping on in how you use AI in your day-to-day work?
在你如何在日常工作中使用人工智能时，有什么真正有趣的事情或者你认为人们会忽视的事情吗？
Kevin Weil: We use it a lot.
凯文·威尔：我们经常使用它。
I mean, every one of us is in Chat GPT all the time summarizing docs, using it to help write docs with GPTs that write product specs and things like that, all the stuff that you would imagine.
我的意思是，我们每个人都一直在 Chat GPT 中总结文档，用它来帮助用 GPT 编写文档，编写产品规格和类似的东西，所有你能想象到的东西。
I mean talk about writing evals, you can actually use models to help you write evals and they're pretty good at it.
我的意思是谈论编写评估，您实际上可以使用模型来帮助您编写评估，并且它们非常擅长。
That all said, I'm still sort of disappointed by us, and I really mean me, in, if I were to just teleport my five-year-old self leading product at some other company into my day job, I would recognize it still.
话虽如此，我仍然对我们感到有点失望，我真的是说我，如果我只是将我在其他公司五年前的自我领先产品转移到我的日常工作中，我仍然会认出它。
And I think we should be in a world, certainly a year from now, probably even more now, where I almost wouldn't recognize it because the workflows are so different and I'm using AI so heavily, and I'd still recognize it today.
我认为我们应该处于一个世界，当然是一年后，甚至可能更久，我几乎认不出它，因为工作流程是如此不同，而且我如此大量地使用人工智能，而且我今天仍然认得它。
So I think in some sense, I'm not doing a good enough job of that.
所以我认为从某种意义上说，我在这方面做得还不够好。
Kevin Weil: Just to give an example, why shouldn't we be vibe coding demos right, left and center?
Kevin Weil：举个例子，为什么我们不应该在右侧、左侧和中心进行氛围编码演示？
Instead of showing stuff in Figma, we should be showing prototypes that people are vibe coding over the course of 30 minutes to illustrate proofs of concept and to explore ideas.
我们不应该在 Figma 中展示东西，而应该展示人们在 30 分钟内进行氛围编码的原型，以说明概念证明并探索想法。
That's totally possible today, and we're not doing it enough.
今天这是完全可能的，但我们做得还不够。
Actually, our chief people officer, Julia, was telling me the other day, she vibe coded an internal tool that she had at a previous job that she really wanted to have here at Open AI and she opened, I don't know, Windsurf or something, and vibe coded it.
事实上，我们的首席人力官朱莉娅前几天告诉我，她对一个内部工具进行了编码，这是她在之前的工作中拥有的，她真的很想在 Open AI 拥有这个工具，她打开了，我不知道，Windsurf 之类的东西，然后对它进行了编码。
How cool is that?
那有多酷？
And if our chief people officer is doing it, we have no excuse to not be doing it more.
如果我们的首席人力官正在这样做，我们就没有理由不做得更多。
Lenny Rachitsky: That's an awesome story.
莱尼·拉奇茨基：这是一个很棒的故事。
And some people may not have heard this term vibe coding.
有些人可能没有听说过氛围编码这个词。
Can you describe what that means?
你能描述一下这意味着什么吗？
Kevin Weil: Yeah, I think this was Andrej's term.
凯文·韦尔：是的，我认为这是安德烈的术语。
Kevin Weil: Andrej Karpathy.
凯文·韦尔：安德烈·卡帕蒂。
Yeah.
是的。
So you have these tools like Cursor and Windsurf and GitHub Copilot that are very good at suggesting what code you might want to write.
因此，您可以使用 Cursor、Windsurf 和 GitHub Copilot 等工具，它们非常适合建议您可能想要编写的代码。
So you can give them a prompt and they'll write code and then as you go to edit it, it's suggesting what you might want to do.
所以你可以给他们一个提示，他们会编写代码，然后当你去编辑它时，它会建议你可能想做的事情。
And the way that everyone started using that stuff was, give it a prompt, have it do stuff, you go edit it, give it a prompt, and you're kind of really going back and forth with the model the whole time.
每个人开始使用这些东西的方式是，给它一个提示，让它做一些事情，你去编辑它，给它一个提示，然后你就一直在模型上来来回回。
As the models are getting better and as people are getting more used to it, you can kind of just let go of the wheel a little bit.
随着模型变得越来越好，人们也越来越习惯它，你可以稍微放开方向盘。
And when the model's suggesting stuff, it's just like, tap, tap, tap, tap, tap.
当模型提出建议时，就像是，点击，点击，点击，点击，点击。
Keep going.
继续前进。
Yes, yes, yes, yes, yes.
是的，是的，是的，是的，是的。
Kevin Weil: And of course the model makes mistakes or it does something that doesn't compile, but when it doesn't compile, you paste the error in and you say, go, go, go, go, go.
Kevin Weil：当然，模型会犯错误，或者做了一些无法编译的事情，但是当它无法编译时，你将错误粘贴进去，然后说，走，走，走，走，走。
And then you test it out and it does one thing that you don't want it to do, so you enter in an instruction and say, go, go, go, go, go, and you just let the model do its thing.
然后你测试它，它做了一件你不希望它做的事情，所以你输入指令并说，走，走，走，走，走，然后你就让模型做它的事情。
And it's not that you would do that for production code that needed to be super tight today yet, but for so many things, you're trying to get to a proof of concept, you're getting to a demo and you can really take your hands off the wheel and the model will do an amazing job, and that's vibe coding.
这并不是说你会为今天需要超级严格的生产代码这样做，但对于很多事情，你试图获得概念证明，你正在进行演示，你可以真正将手从方向盘上移开，模型将完成出色的工作，这就是氛围编码。
Lenny Rachitsky: That's an awesome explanation.
Lenny Rachitsky：这是一个很棒的解释。
I think the pro version of that, which is, I think, the way Andre even described it as you talk, there's a step like whisper or super whisper or something like that where you're talking to the model, not even typing.
我认为专业版本，我认为，安德烈甚至在你说话时描述它的方式，有一个像耳语或超级耳语或类似的步骤，你在与模型交谈，甚至不打字。
Lenny Rachitsky: Oh man.
莱尼·拉奇茨基：天啊。
So let me just ask, I guess, when you look at product teams in the future, you talked about how you guys should be doing this more, instead of designs, having prototypes, what do you think might be the biggest changes in how product teams are structured or built?
所以我想问一下，我想，当你在未来看待产品团队时，你谈到了你们应该如何更多地做这件事，而不是设计、拥有原型，你认为产品团队的结构或构建方式最大的变化可能是什么？
Where do you think things are going in the next few years?
您认为未来几年事情会走向何方？
Kevin Weil: I think you're definitely going to live in a world where you have researchers built into every product team.
Kevin Weil：我认为你肯定会生活在一个每个产品团队都有研究人员的世界。
And I don't even mean just at foundation model companies because I think the future...
我什至不仅仅指基础模型公司，因为我认为未来......
Actually, frankly one thing that I'm sort of surprised about about our industry in general is that there's not a greater use of fine-tuned models.
事实上，坦率地说，我对我们整个行业感到有点惊讶的一件事是，没有更多地使用微调模型。
A lot of people...
很多人...
These models are very good, so our API does a lot of things really well, but when you have particular use cases, you can always make the model perform better on a particular use case by fine-tuning it.
这些模型非常好，因此我们的 API 在很多事情上做得非常好，但是当您有特定的用例时，您始终可以通过微调模型来使模型在特定用例上表现更好。
It's probably just a matter of time.
这可能只是时间问题。
Folks aren't quite comfortable yet with doing that in every case.
人们还不太习惯在所有情况下都这样做。
But to me, there's no question that that's the future.
但对我来说，毫无疑问这就是未来。
Models are going to be everywhere just like transistors are everywhere, AI is going to be just a part of the fabric of everything we do, but I think there are going to be a lot of fine-tuned models because why would you not want to more specifically customize a model against a particular use case?
模型将无处不在，就像晶体管无处不在一样，人工智能将只是我们所做的一切结构的一部分，但我认为将会有很多经过微调的模型，因为为什么你不想针对特定用例更具体地定制模型呢？
Kevin Weil: And so I think you're going to want sort of quasi researcher machine learning engineer types as part of pretty much every team because fine-tuning a model is just going to be part of the core workflow for building most products.
Kevin Weil：所以我认为几乎每个团队都需要准研究员机器学习工程师类型，因为微调模型将成为构建大多数产品的核心工作流程的一部分。
So that's one change that maybe you're starting to see at foundation model companies that will propagate out to more teams over time.
因此，这是您可能开始在基础模型公司看到的一项变化，随着时间的推移，这种变化将传播到更多团队。
Lenny Rachitsky: I'm curious if there's a concrete example that makes that real, and I'll share one that comes to mind as you talk, which is, when you look at Cursor and Windsurf, something I learned from those founders is that they use a Sonnet, but then they also have a bunch of custom models that help along the edges that make the specific experience that's not just generating code even better like auto-complete and looking ahead to where things are going.
Lenny Rachitsky：我很好奇是否有一个具体的例子可以证明这一点，我会分享你在谈论时想到的一个例子，那就是，当你看到 Cursor 和 Windsurf 时，我从那些创始人那里学到的东西是他们使用十四行诗，但他们也有一堆自定义模型，可以帮助沿着边缘，使特定的体验不仅仅是生成代码甚至更好，比如自动完成和展望事情的发展方向。
So is that one or any other examples of which you...
这就是您所举的一个或任何其他例子吗？
What is a fine-tuned model?
什么是微调模型？
Do you think teams will be building with these researchers on their teams?
您认为团队会与这些研究人员一起构建团队吗？
Kevin Weil: Yeah.
凯文·威尔：是的。
I mean, so when you're a model, you're basically giving the model a bunch of examples of the kinds of things you want it to be better at.
我的意思是，当你是一个模型时，你基本上是在给模型提供一堆你希望它擅长的事情的例子。
So it's, "Here's a problem, here's a good answer.
所以，“这是一个问题，这是一个很好的答案。
Here's a problem, here's a good answer," Or, "Here's a question, here's a good answer times a thousand or 10,000." And suddenly you're teaching the model to be much better than it was out of the gate at that particular thing.
这是一个问题，这是一个好的答案，”或者，“这是一个问题，这是一个好的答案乘以一千或一万倍。”突然之间，你教授的模型在特定的事情上比刚开始时要好得多。
We use it everywhere internally.
我们在内部到处都使用它。
We use ensembles of models much more internally than people might think.
我们在内部使用模型集合的次数比人们想象的要多得多。
So it's not, "I have 10 different problems.
所以并不是，“我有 10 个不同的问题。
I'll just ask baseline GPT four oh about a bunch of these things." If we have 10 different problems, we might solve them using 20 different model calls, some of which are using specialized fine-tuned models, they're using models of different sizes because maybe you have different latency requirements or cost requirements for different questions.
我只会向基准 GPT 4 询问这些事情。”如果我们有 10 个不同的问题，我们可能会使用 20 个不同的模型调用来解决它们，其中一些使用专门的微调模型，它们使用不同大小的模型，因为也许您对不同的问题有不同的延迟要求或成本要求。
Kevin Weil: They are probably using custom prompts for each one.
Kevin Weil：他们可能对每个提示都使用自定义提示。
Basically you want to teach the model to be really good at...
基本上你想教模型真正擅长......
You want to break the problem down into more specific tasks versus some broader set of high level tasks.
您希望将问题分解为更具体的任务，而不是更广泛的高级任务。
And then you can use models very specifically to get very good at each individual thing.
然后你可以非常具体地使用模型来很好地处理每件事。
And then you have an ensemble that tackles the whole thing.
然后你就有了一个解决整个问题的整体。
I think a lot of good companies are doing that today.
我认为今天很多优秀的公司都在这样做。
I still see a lot of companies giving the model single, generic, broad problems versus breaking the problem down, and I think there will be more breaking the problem down using specific models for specific things, including fine tuning.
我仍然看到很多公司给出的模型是单一的、通用的、广泛的问题，而不是分解问题，而且我认为会有更多的公司使用针对特定事物的特定模型来分解问题，包括微调。
Lenny Rachitsky: And so in your case, because this is really interesting, is that you're using different levels of Chat GPT, like a 1 0 3 and stuff that's earlier because it's cheaper.
Lenny Rachitsky：在你的例子中，因为这真的很有趣，你正在使用不同级别的 Chat GPT，比如 1 0 3 和更早的东西，因为它更便宜。
Kevin Weil: There'll be parts of our internal stack.
Kevin Weil：这将是我们内部堆栈的一部分。
I'll give you an example.
我给你举个例子。
Customer support, with 400 plus million weekly active users, we get a lot of inbound tickets.
客户支持，每周活跃用户超过 400 亿，我们获得了大量入站门票。
I don't know how many customer support folks we have, but it's not very many, 30, 40, I'm not sure, way smaller than you would have at any comparable company, and it's because we've automated a lot of our flows.
我不知道我们有多少客户支持人员，但不是很多，30、40，我不确定，比任何同类公司的数量要少得多，这是因为我们已经实现了很多流程的自动化。
We've got most questions using our internal resources, knowledge base, guidelines for how we answer questions, what kind of personality, et cetera.
我们利用我们的内部资源、知识库、如何回答问题的指南、什么样的个性等等来解决大多数问题。
You can teach the model those things and then have it do a lot of its answers automatically, or where it doesn't have the full confidence to answer a particular question, it can still suggest an answer, request a human to look at it and then that human's answer actually is its own sort of fine tuning data for the model.
你可以教模型这些东西，然后让它自动做很多答案，或者在它没有完全信心回答特定问题的情况下，它仍然可以建议一个答案，要求人类查看它，然后人类的答案实际上是它自己的模型微调数据。
You're telling it the right answer in a particular case.
你告诉它特定情况下的正确答案。
Kevin Weil: We're using...
凯文·威尔：我们正在使用......
At various places.
在不同的地方。
Some of these places, you want a little bit more reasoning, is not super latency sensitive, so you want a little more reasoning, and we'll use one of our O series models.
其中一些地方，您想要更多一点推理，对延迟不敏感，所以您想要更多一点推理，我们将使用我们的 O 系列模型之一。
In other places, you want a quick check on something and so you're fine to use four oh mini, which is super fast and super cheap.
在其他地方，你想要快速检查某些东西，所以你可以使用四哦迷你，它超级快而且超级便宜。
In general, it's like specific models for specific purposes and then you ensemble them together to solve problems.
一般来说，它就像用于特定目的的特定模型，然后将它们组合在一起来解决问题。
By the way, again, not unlike how we as humans solve problems, a company is arguably an ensemble of models that have all been fine tuned based on what we studied in college and what we have learned over the course of our careers.
顺便说一句，与我们人类解决问题的方式没有什么不同，公司可以说是一个模型的集合，这些模型都根据我们在大学学习的知识和我们在职业生涯中学到的知识进行了微调。
We've all been fine tuned to have different sets of skills and you group them together in different configurations and the output of the ensemble is much better than the output of any one individual.
我们都经过精心调整，拥有不同的技能组合，并且将它们以不同的配置组合在一起，并且整体的输出比任何一个个体的输出都要好得多。
Lenny Rachitsky: Kevin, you're blowing my mind.
莱尼·拉奇茨基：凯文，你让我大吃一惊。
That sounds exactly correct.
这听起来完全正确。
And also, different people, you pay them less, they cost less to talk to, some people take a long time to answer, some people hallucinating.
而且，对于不同的人，你付给他们的钱更少，与他们交谈的成本也更少，有些人需要很长时间才能回答，有些人会产生幻觉。
This is...
这是...
Kevin Weil: I'm telling you.
凯文·威尔：我告诉你。
This is a mental model but really does work in thinking...
这是一种心理模型，但确实适用于思考......
Lenny Rachitsky: Oh, right.
莱尼·拉奇茨基：哦，对了。
Yeah.
是的。
This is great.
这太棒了。
Some people are visual, they want to dry out their thinking, some people want to talk word cell.
有些人是视觉的，他们想要干燥他们的思维，有些人想要谈论文字细胞。
Wow, this is a really good metaphor.
哇，这真是一个很好的比喻。
So again, coming back to your advice here because I love that we circled back to it, you're finding a really good way to think about how to design great AI experiences and LMs, I guess, specifically is think about how a person would do this.
再次回到你的建议，因为我喜欢我们回过头来，你找到了一个非常好的方法来思考如何设计出色的 AI 体验和 LM，我想，具体来说就是考虑一个人会如何做到这一点。
Kevin Weil: Well, it's maybe not always the answer is to think about how a person would do it, but sometimes to gain intuition for how you might solve a problem, you think about what an equivalent human would do in those situations and use that to at least gain a different perspective on the problem.
凯文·韦尔：嗯，也许并不总是问题的答案是思考一个人会如何做，但有时为了获得如何解决问题的直觉，你会思考一个同等的人在这些情况下会做什么，并利用它至少获得对问题的不同视角。
Kevin Weil: Because some of this really is talking to a model.
凯文·威尔：因为其中一些确实是在与模特交谈。
There's a lot of prior art because we talk to other humans all the time and encounter them in all sorts of different situations, and so there's a lot to learn from that.
有很多现有技术，因为我们一直与其他人交谈并在各种不同的情况下遇到他们，所以有很多东西可以从中学习。
Lenny Rachitsky: Okay, so speaking of humans, I want to chat about the future a little bit.
Lenny Rachitsky：好的，说到人类，我想谈谈未来。
So you have three kids, and a community member asked me this hilarious question that I think it's something a lot of people are thinking about.
你有三个孩子，一位社区成员问了我这个有趣的问题，我认为这是很多人都在思考的问题。
So this is Patrick .
这是帕特里克。
I worked with him at Airbnb.
我和他一起在 Airbnb 工作。
He says ask what he's encouraging his kids to learn to prepare for the future.
他说，问问他鼓励孩子们学习什么，为未来做好准备。
I'm worried my 6-year-old by the year 2036 will face a lot of competition trying to get into the top roofing or plumbing programs and need a backup plan.
我担心到 2036 年，我 6 岁的孩子将面临很多竞争，试图进入顶级屋顶或管道项目，并需要一个备用计划。
Kevin Weil: That's funny.
凯文·威尔：这很有趣。
So our kids, we have a 10 year old and eight year old twins, so they're still pretty young.
我们的孩子，我们有一个 10 岁和 8 岁的双胞胎，所以他们还很年轻。
It's amazing how AI native they are.
令人惊奇的是它们是人工智能原生的。
It's completely normal to them that there are self-driving cars.
对他们来说，有自动驾驶汽车是完全正常的。
That they can talk to AI all day long.
他们可以整天与人工智能交谈。
They have full conversations with Chat GPT and Alexa and everything else.
他们与 Chat GPT 和 Alexa 以及其他一切进行完整的对话。
I don't know, who knows what the future holds?
我不知道，谁知道未来会怎样？
I think things like coding skills are going to be relevant for a long time, who knows?
我认为编码技能之类的东西将在很长一段时间内发挥作用，谁知道呢？
But I think if you teach your kids to be curious, to be independent, to be self-confident, you teach them how to think, I don't know what the future holds, but I think that those are going to be skills that are going to be important in any configuration of the future.
但我认为，如果你教你的孩子保持好奇心、独立、自信，教他们如何思考，我不知道未来会怎样，但我认为这些技能在未来的任何配置中都将是重要的。
And so it's not like we have all the answers, but that's how Elizabeth and I think about our kids.
所以我们并不知道所有的答案，但这就是伊丽莎白和我对我们孩子的看法。
Lenny Rachitsky: And do you find that AI...
Lenny Rachitsky：你有没有发现人工智能......
There's a lot of talk about AI tutoring.
关于人工智能辅导有很多讨论。
Is that something you guys are doing?
这是你们正在做的事情吗？
I know they're using Chat GPT, I love all the photos you post where they're playing with prompts and stuff, but I guess is there anything there you're experimenting with or you think is going to become really important?
我知道他们正在使用 Chat GPT，我喜欢你发布的所有他们在玩提示和其他东西的照片，但我想你正在尝试什么或者你认为会变得非常重要吗？
Kevin Weil: This is something that...
凯文·威尔：这是……
It's maybe the most important thing that AI could do.
这可能是人工智能可以做的最重要的事情。
Maybe that's a grand statement.
也许这是一个伟大的声明。
There are lots of important things that AI can do, including speeding up the pace of fundamental science research and discovery, which maybe is actually the most important thing AI can do.
人工智能可以做很多重要的事情，包括加快基础科学研究和发现的步伐，这实际上可能是人工智能可以做的最重要的事情。
But one of the most important things would be personalized tutoring.
但最重要的事情之一是个性化辅导。
And it kind of blows my mind that there is still...
令我震惊的是，仍然有......
I know there are a bunch of good products out there.
我知道那里有很多好产品。
Khan Academy does great things.
可汗学院做了伟大的事情。
They're a wonderful partner of ours.
他们是我们的出色合作伙伴。
Vinod Khosla has a non-profit that's doing some really interesting stuff in this space and is making an impact.
Vinod Khosla 有一个非营利组织，在这个领域做了一些非常有趣的事情，并且正在产生影响。
But I'm kind of surprised that there isn't a 2 billion kid AI personalized tutoring thing because the models are good enough to do it now, and every study out there that's ever been done seems to show that when you have...
但令我有点惊讶的是，还没有针对 20 亿儿童的人工智能个性化辅导，因为现在的模型已经足够好，可以做到这一点，而且每项研究似乎都表明，当你...
Like, education is still important, but when you combine that with personalized tutoring, you get multiple standard deviation improvements in learning speed.
就像，教育仍然很重要，但是当你将其与个性化辅导结合起来时，你的学习速度会得到多个标准差的提高。
Kevin Weil: And so it's uncontroversial, it's good for kids, it's free.
凯文·韦尔：所以这是没有争议的，它对孩子们有好处，而且是免费的。
Chat GPT is free, you don't need to pay, and the models are good enough.
聊天GPT是免费的，不需要付费，而且模型足够好。
It still just kind of blows my mind that there isn't something amazing out there that our kids are using and your future kids are using, and people in all sorts of places around the world that aren't as lucky as our kids to be able to have this sort of built-in, solid education.
仍然让我感到震惊的是，我们的孩子和你未来的孩子正在使用的东西并没有什么令人惊奇的，世界各地的人们都没有我们的孩子那么幸运能够接受这种内置的、扎实的教育。
Again, Chat GPT is free.
同样，Chat GPT 是免费的。
People have Android devices everywhere.
人们到处都有 Android 设备。
I really just think this could change the world and I'm surprised it doesn't exist and I want it to exist.
我真的只是认为这可以改变世界，但我很惊讶它不存在，但我希望它存在。
Lenny Rachitsky: This kind of touches on something I want to spend a little time on, which is a lot of people also worry a lot about AI, where it's going, they worry about jobs it's going to take, they worry about the super intelligence squashing humanity in the future.
Lenny Rachitsky：这涉及到我想花一点时间的事情，即很多人也非常担心人工智能，它的发展方向，他们担心它会抢走工作，他们担心未来超级智能会压垮人类。
What's your perspective on that and just the optimistic case that I think people need to hear?
您对此有何看法？我认为人们需要听到的乐观案例是什么？
Kevin Weil: I mean, I'm a big technology optimist.
凯文·韦尔：我的意思是，我是一位技术乐观主义者。
I think if you look over the last 200 years, maybe more, technology has driven a lot of the advancements that have made us the world and the society that we are today.
我认为，如果你回顾过去 200 年，也许更久，技术推动了许多进步，使我们成为今天的世界和社会。
It drives economic advancements, it drives geopolitical advancements, quality of life, longevity advancement.
它推动经济进步、地缘政治进步、生活质量和寿命的进步。
I mean, technology's at the root of just about everything, so I think there are very few examples where this is anything but a great thing over the longer term.
我的意思是，技术是一切事物的根源，所以我认为从长远来看，很少有例子表明这不是一件好事。
That doesn't mean that there aren't...
这并不意味着没有...
Kevin Weil: ...
凯文·威尔：...
a great thing over the longer term.
从长远来看，这是一件伟大的事情。
That doesn't mean that there aren't temporary dislocations or where there aren't individuals that are impacted, and that matters too.
这并不意味着不存在暂时的混乱或没有个人受到影响，这也很重要。
So it can't just be that the average is good.
所以不能仅仅因为平均水平就很好。
You've got to also think about how you take care of each individual person as best you can.
您还必须考虑如何尽力照顾每个人。
Kevin Weil: It is something that we think a lot about and as we work with the administration, as we work with policy, we try and help wherever we can.
凯文·威尔：这是我们经常思考的事情，当我们与政府合作、制定政策时，我们会尽力提供帮助。
We do a lot with education.
我们在教育方面做了很多工作。
One of the benefits here is that ChatGPT is also perhaps the best reskilling app you could possibly want.
这里的好处之一是 ChatGPT 或许也是您可能想要的最好的技能再培训应用程序。
It knows a lot of things.
它知道很多事情。
It can teach you a lot of things if you're interested in learning new things.
如果您有兴趣学习新事物，它可以教您很多东西。
Kevin Weil: These are very real issues.
凯文·威尔：这些都是非常现实的问题。
I'm super optimistic about the long run, and we're going to need to do everything we can as a society to ensure that we make this transition as graceful and as well-supported as we can.
从长远来看，我非常乐观，作为一个社会，我们需要尽一切努力，确保我们能够顺利完成这一过渡，并尽可能得到良好的支持。
Lenny Rachitsky: To give people a sense of where things might be going.
Lenny Rachitsky：让人们了解事情可能的发展方向。
That's a big question in a lot of people's minds.
这是很多人心中的一个大问题。
So someone asked this question that I love, which is, "AI is already changing, creative work in a lot of different ways, writing and design and coding, what do you think is the next big leap?
所以有人问了我喜欢的这个问题，即“人工智能已经在以很多不同的方式改变创造性的工作，写作、设计和编码，你认为下一个巨大的飞跃是什么？
What should we be thinking is the next big leap in AI-assisted creativity specifically, and then just broadly, where do you think things are going to be going in the next few years?"
我们应该思考的是人工智能辅助创造力的下一个重大飞跃，具体而言，然后从广义上讲，你认为未来几年事情会走向何方？”
Kevin Weil: Yeah.
凯文·威尔：是的。
This is also an area where I'm a big optimist.
这也是我非常乐观的一个领域。
If you look at Sora, for example.
例如，如果你看看索拉。
I mean we talked about ImageGen earlier and the absolute fount of creativity that people are putting across Twitter and Instagram and other places.
我的意思是，我们之前讨论过 ImageGen 以及人们在 Twitter、Instagram 和其他地方发布的绝对创造力源泉。
I am the world's worst artist like the worst.
我是世界上最糟糕的艺术家。
Maybe the only thing I'm worse at than art is singing.
也许我唯一比艺术差的就是唱歌。
Give me a pencil and a pad of paper and I can't draw better than our eight-year-old.
给我一支铅笔和一张纸，我画得也不会比我们八岁的孩子画得好。
But give me ImageGen and I can think some creative thoughts and put something into the model and suddenly have output that I couldn't have possibly done myself.
但是给我 ImageGen，我就可以想出一些创造性的想法，并将一些东西放入模型中，然后突然得到我自己不可能完成的输出。
That's pretty cool.
太酷了。
Kevin Weil: Even you look at folks that are really talented.
凯文·威尔：即使是你也会看到那些真正有才华的人。
I was talking to a director recently about Sora, someone who's directed films that we would all know, and he was saying, for a film that he's doing, take the example of some sort of sci-fi-ish, think of Star Wars, and you've got some scene where there's a plane zooming into some Death Star-like thing.
我最近和一位导演谈论了索拉，他执导过我们都知道的电影，他说，对于他正在制作的电影，以某种科幻小说为例，想想星球大战，你会看到一些场景，其中有一架飞机飞向某个类似死星的物体。
And so you've got the plane looking at the whole planet, and then you want to cut to a scene where the plane's kind of at the ground level, and all of a sudden you see the city and everything else.
所以你让飞机看着整个星球，然后你想切到飞机在地面上的场景，突然你就看到了城市和其他一切。
How are we going to manage that cut scene?
我们要如何管理这个过场动画？
And that transition?
那过渡呢？
Kevin Weil: And he was saying, "In the world of two years ago, I would have paid a 3D effects company a hundred grand and they would've taken a month, and they would've produced two versions of this cut scene for me.
凯文·威尔：他说，“在两年前，我会向 3D 特效公司支付一百英镑，他们会花一个月的时间，为我制作这个过场动画的两个版本。
And I would've evaluated them.
我会对他们进行评估。
We would've chosen one, because what are you going to do?
我们会选择其中之一，因为你要做什么？
Pay another 50 grand and wait another month.
再付50块钱，再等一个月。
And we would've just gone with it.
我们就会接受它。
And it would be fine.
那就好了。
Movies are great.
电影很棒。
I love them.
我爱他们。
And there've been..."
而且已经有……”
Kevin Weil: Obviously, we can do great things with the technology that we've had, but you now look at what you can do with Sora.
Kevin Weil：显然，我们可以利用我们现有的技术做出伟大的事情，但你现在看看你可以用 Sora 做什么。
And his point was, "Now, I can use Sora, our video model, and I can get 50 different variations of this cut scene just me brainstorming into a prompt and the model brainstorming a little bit with me.
他的观点是，“现在，我可以使用 Sora，我们的视频模型，我可以得到这个过场动画的 50 种不同变体，只需我集思广益形成提示，模型与我一起集思广益。
I've got 50 different versions.
我有 50 个不同的版本。
And then of course, I can iterate off of those and refine them and take different ideas.
当然，我可以迭代这些并完善它们并采取不同的想法。
And now I'm still going to go to that 3D effects studio to produce the final one, but I'm going to go having brainstormed and had a much more creative approach with an outcome that's much better.
现在我仍然会去那个 3D 效果工作室制作最后一个，但我会进行集思广益，并采取更有创意的方法，以获得更好的结果。
And I did that assisted by AI."
我在人工智能的帮助下做到了这一点。”
Kevin Weil: My personal view on creativity in general is that it's no one's going to...
凯文·威尔：我个人对创造力的看法是，没有人会……
You don't type into Sora like, "Make me a great movie." It requires creativity and ingenuity, and all these things, but it can help you explore more.
你不会在 Sora 中输入“让我拍一部很棒的电影”之类的内容。它需要创造力和独创性，以及所有这些东西，但它可以帮助你探索更多。
It can help you get to a better final result.
它可以帮助您获得更好的最终结果。
So, again, I tend to be an optimist in most things, but actually, I think there's a very good story here.
所以，再说一次，我在大多数事情上都是乐观主义者，但实际上，我认为这里有一个非常好的故事。
Lenny Rachitsky: I know Sam Altman, I think it was him who tweeted recently, the creative writing piece that you guys are working on where it's...
Lenny Rachitsky：我认识 Sam Altman，我想是他最近发了推文，你们正在创作的创意写作作品……
He is very bad at writing creative stuff, and he shared an example where it's actually really good.
他不擅长写有创意的东西，但他分享了一个实际上非常好的例子。
I imagine that's another area of investment.
我想这是另一个投资领域。
Kevin Weil: Yeah, there's some exciting stuff happening internally with some new research techniques.
Kevin Weil：是的，一些新的研究技术在内部发生了一些令人兴奋的事情。
We'll have more to say about that at some point.
我们将在某个时候对此进行更多讨论。
But yeah, Sam sometimes likes to show off some of the stuff that's coming, which is smart.
但是，是的，山姆有时喜欢炫耀一些即将推出的东西，这很聪明。
By the way, it's very indicative of this iterative deployment philosophy.
顺便说一句，它非常体现了这种迭代部署理念。
We don't have some breakthrough and keep it to ourselves forever, and then bestow it upon the world someday.
我们没有什么突破，永远把它留给自己，然后有一天把它送给世界。
We kind of just talk about the things we're working on and share when we can and launch early and often, and then iterate in public.
我们只是谈论我们正在做的事情，并在可能的情况下分享，并尽早且频繁地发布，然后公开迭代。
I really like that philosophy.
我真的很喜欢这种哲学。
Lenny Rachitsky: I love all these hints that a few things coming.
Lenny Rachitsky：我喜欢所有这些暗示即将发生的事情。
I know you can't say too much.
我知道你不能说太多。
You talked about how there might be a coding leap coming in the near future maybe by the time this comes out.
你谈到了在不久的将来可能会出现编码飞跃，也许到了这个发布的时候。
Is there anything else people should be thinking about, might be coming in the near future?
在不久的将来，还有什么人们应该考虑的事情吗？
Any things you can tease that are interesting?
有什么有趣的事情可以调侃吗？
Exciting?
令人兴奋吗？
Kevin Weil: Man, this hasn't been enough for you?
凯文·威尔：伙计，这对你来说还不够吗？
Lenny Rachitsky: Only everything is getting better every day.
Lenny Rachitsky：一切都在一天比一天好。
Kevin Weil: Yeah.
凯文·威尔：是的。
I'm like, man, I hope we get some of this stuff out before the episode launches so-
我想，伙计，我希望我们能在剧集开始之前把这些东西拿出来，所以-
Kevin Weil: ...
凯文·威尔：...
I don't piss people off.
我不惹恼别人。
The amazing thing to me is we were talking earlier about how far models have come in just a couple of years.
对我来说，令人惊奇的是我们之前讨论过模型在短短几年内取得了多大的进步。
If you went back to GPT-3, you'd be disgusted by how bad it was, even though Lenny of two years ago was mind-blown by how good these were.
如果你回到 GPT-3，你会对它的糟糕程度感到厌恶，尽管两年前的 Lenny 还对它们的出色表现感到震惊。
And for a long time, we were iterating every six to nine months on a new GPT model.
在很长一段时间里，我们每六到九个月就会迭代一个新的 GPT 模型。
It was like GPT-3, GPT-3.5, 4, and now with this o-series of reasoning models, we're moving even faster.
就像 GPT-3、GPT-3.5、4，现在有了这个 O 系列推理模型，我们的进展更快。
Every roughly three months, maybe four months, there's a new o-series model, and each of them is a step up in capability.
大约每隔三个月，也许四个月，就会有一款新的 O 系列型号推出，每款型号的功能都得到了提升。
Kevin Weil: And so the capabilities of these models are increasing at a massive pace.
Kevin Weil：因此这些模型的功能正在飞速增长。
They're also getting cheaper as they scale.
随着规模的扩大，它们的价格也变得越来越便宜。
You look at where we were even a couple of years ago.
你看看我们几年前的处境。
I think the original, I don't know, what was it, GPT-3.5 or something was like 100 x the cost of GPT-4o mini today in the API.
我认为最初的，我不知道，它是什么，GPT-3.5 之类的东西在 API 中的成本大约是 GPT-4o mini 成本的 100 倍。
A couple of years, you've gone down two orders of magnitude in cost for much more intelligence.
几年来，您已经将成本降低了两个数量级，以获得更多的智能。
And so I don't know where there's another series of trends like that in the world.
所以我不知道世界上哪里还有类似的一系列趋势。
Models are getting smarter, they're getting faster, they're getting cheaper, and they're getting safer too.
模型变得越来越智能，速度越来越快，价格越来越便宜，而且也越来越安全。
They hallucinate less every iteration.
每次迭代他们产生的幻觉都会减少。
Kevin Weil: And so the Morse Law and transistors becoming ubiquitous.
凯文·威尔：因此莫尔斯定律和晶体管变得无处不在。
That was a law around doubling the number of transistors on a chip every 18 months.
这是一条关于每 18 个月将芯片上的晶体管数量增加一倍的法律。
If you're talking about something where you're getting 10 x every year, that's a massively steeper exponential.
如果你谈论的是每年获得 10 倍的回报，那么这是一个陡峭得多的指数。
And it tells us that the future is going to be very different than today.
它告诉我们，未来将与今天有很大不同。
The thing I try and remind myself is, the AI models that you're using today is the worst AI model you will ever use for the rest of your life.
我试图提醒自己的是，你今天使用的人工智能模型是你余生中使用过的最糟糕的人工智能模型。
And when you actually get that in your head, it's kind of wild.
当你真正想到这一点时，你会发现它有点疯狂。
Lenny Rachitsky: I was going to actually say the same thing, and that's the thing that always sticks with me when I watch this thing.
Lenny Rachitsky：我其实也想说同样的话，当我看到这件事时，这件事一直萦绕在我心头。
You're talking about Sora, and I imagine many people hearing that are like, "No, no. It's not actually ready.
你在谈论索拉，我想很多人听到这句话都会说，“不，不。它实际上还没有准备好。
It's not good enough.
这还不够好。
It's not going to be as good as a movie I see in the theater." But the point is what you just made that this is the worst it's going to be.
它不会像我在剧院看到的电影那么好。”但重点是你刚才所说的，这将是最糟糕的情况。
It will only get better.
只会变得更好。
Kevin Weil: Yeah, model maximalism.
凯文·威尔：是的，极简主义典范。
Just keep building for the capabilities that are almost there, and the model's going to catch up and be amazing.
只要继续构建几乎已经具备的功能，模型就会迎头赶上并令人惊叹。
Lenny Rachitsky: Escape to where the puck is going to be.
Lenny Rachitsky：逃到冰球所在的地方。
Lenny Rachitsky: This reminds me, I was just using...
Lenny Rachitsky：这提醒了我，我只是在使用......
I was duplifying everything the other day and I was just like, "What is taking so long."
前几天我把所有的事情都重复了，我就想，“怎么花了这么长时间。”
Lenny Rachitsky: Just like cut...
莱尼·拉奇茨基：就像剪辑一样……
What was that?
那是什么？
Lenny Rachitsky: As one does these days.
Lenny Rachitsky：正如人们现在所做的那样。
I was just like, "It's taking a minute to generate this image of my family in this amazing way." Come on, what's taking so long.
我当时想，“花了一分钟的时间才能以这种令人惊奇的方式生成我的家庭形象。”来吧，什么事要花这么长时间。
You just get so used to magic happening in front of you.
你只是已经习惯了发生在你面前的魔法。
Lenny Rachitsky: Okay, final question.
莱尼·拉奇茨基：好的，最后一个问题。
This is going to go in a completely different direction.
这将朝着完全不同的方向发展。
A lot of people asked about this.
很多人问过这个问题。
So famously, you led this project at Facebook called Libra, which is now called Novi.
众所周知，您在 Facebook 领导了这个名为 Libra 的项目，该项目现在称为 Novi。
A lot of people always wondered, "What happened there?
很多人总是想知道：“那里发生了什么？
That was a really cool idea." I know some people have a sense there's regulation challenges, things like that.
这真是一个很酷的主意。”我知道有些人感觉到存在监管挑战之类的问题。
I don't know if you've talked about this much.
不知道你讲了这么多没有。
So I guess, could you just give people a brief summary of just what is Libra?
所以我想，您能给人们简单介绍一下什么是 Libra 吗？
This project you working on, and just what happened, and how you feel about it?
你正在从事的这个项目，到底发生了什么，你对此有何感想？
Kevin Weil: Yeah.
凯文·威尔：是的。
I mean, David Marcus led it, and I happily work for him and with him.
我的意思是，大卫·马库斯领导了这一切，我很高兴为他工作并与他一起工作。
I think he's a visionary and also a mentor and a friend.
我认为他是一位有远见的人，也是一位良师益友。
Honestly, Libra is probably the biggest disappointment of my career.
老实说，Libra 可能是我职业生涯中最大的失望。
When I think about the problems we were solving, which are very real problems.
当我想到我们正在解决的问题时，这些都是非常现实的问题。
If you look at, for example, the remittance space, people sending money to family members in other countries, it is maybe...
例如，如果你看看汇款空间，人们向其他国家的家庭成员汇款，这可能是......
I mean it's incredibly regressive, right?
我的意思是这是令人难以置信的倒退，对吧？
People that don't have the money to spend are having to pay 20% to send money home to their family.
没有钱花的人必须支付 20% 的钱才能寄回家给家人。
So outrageous fees, it takes multiple days, you have to go then pick up cash from...
费用太离谱了，需要好几天，你必须去那里领取现金。
It's all bad.
一切都很糟糕。
Kevin Weil: And here we are with 3 billion people using WhatsApp all over the world, talking to each other every day, especially friends and family, and exactly the kind of people who'd send money to each other.
Kevin Weil：全世界有 30 亿人使用 WhatsApp，每天互相交谈，尤其是朋友和家人，而且正是那些会互相汇款的人。
Why can't you send money as immediately, as cheaply, as simply as you send a text message?
为什么您不能像发送短信那样立即、便宜、简单地汇款？
It is one of those things when you sit back and think about it, that should just exist.
当你坐下来思考时，它就是其中之一，它应该存在。
And that was what we set out to try and do.
这就是我们打算尝试做的事情。
Kevin Weil: Now, I don't think we played all of our cards perfectly.
凯文·威尔：现在，我认为我们并没有完美地发挥出所有的牌。
If I could go back and do things, there are a bunch of things I would do differently.
如果我可以回去做一些事情，我会做很多不同的事情。
Kevin Weil: We tried to get it all at once.
凯文·威尔：我们试图一次性完成这一切。
We tried to launch a new blockchain.
我们尝试推出一个新的区块链。
It was a basket of currencies originally.
最初是一篮子货币。
It was integration into WhatsApp and Messenger, and I think the whole world kind of went like, "Oh my God, that's a lot of change at once." And it happened also to be at the time that Facebook was at the absolute nadir of its reputation.
它被集成到 WhatsApp 和 Messenger 中，我认为整个世界都像是，“天哪，这一下子就发生了很大的变化。”恰巧此时 Facebook 的声誉正处于最低谷。
And so that didn't help.
所以这没有帮助。
It was also not the Messenger that people wanted for this kind of change.
人们想要进行这种改变的也不是信使。
We knew all that going in, but we went for it.
我们知道发生的一切，但我们还是坚持了下来。
Kevin Weil: I think there are a bunch of ways that we could do that that would've introduced the change a little bit more gently, maybe still gotten to that same outcome, but fewer new things at once and introduced the new things one at a time.
凯文·威尔：我认为我们可以通过很多方法来更温和地引入变革，也许仍然能达到相同的结果，但同时减少新事物，并一次引入一个新事物。
Who knows?
谁知道？
Those were decisions we made together.
这些是我们共同做出的决定。
So we all own them.
所以我们都拥有它们。
Certainly, I own them.
当然，我拥有它们。
But it fundamentally disappoints me that this doesn't exist in the world today because the world would be a better place if we'd been able to ship that product.
但让我从根本上感到失望的是，当今世界上不存在这种情况，因为如果我们能够运送该产品，世界将会变得更加美好。
I would be able to send you 50 cents in WhatsApp for free.
我可以通过 WhatsApp 免费向您发送 50 美分。
It would settle instantly.
它会立即解决。
Everybody would have a balance in their WhatsApp account.
每个人的 WhatsApp 帐户中都会有余额。
We'd be transact...
我们本来可以交易...
I mean, it should exist.
我的意思是，它应该存在。
Kevin Weil: I don't know.
凯文·威尔：我不知道。
To be honest, the current administration is super friendly to crypto.
老实说，现任政府对加密货币非常友好。
Facebook's reputation, Meta's reputation is in a very different place.
Facebook 的声誉和 Meta 的声誉处于非常不同的位置。
Maybe they should go build it now.
也许他们现在应该去建造它。
Lenny Rachitsky: I was looking at the history of it, and apparently, they sold the tech to some private equity company for 200 million bucks.
Lenny Rachitsky：我正在研究它的历史，显然，他们以 2 亿美元的价格将这项技术卖给了一些私募股权公司。
Kevin Weil: There are a couple of current blockchains that are built on the tech because the tech was open-sourced from the beginning.
Kevin Weil：目前有几个区块链是基于该技术构建的，因为该技术从一开始就是开源的。
Aptos and Mistin are two companies that are built off of this tech.
Aptos 和 Mistin 是两家基于这项技术建立的公司。
So at least all of the work that we did, did not die and lives on in these two companies, and they're both doing really well.
因此，至少我们所做的所有工作都没有消亡，而是在这两家公司继续存在，而且它们都做得非常好。
But still, we should be able to send each other money in WhatsApp, and we can't today.
但我们仍然应该能够通过 WhatsApp 互相汇款，但现在还不能。
Lenny Rachitsky: Hear, hear.
莱尼·拉奇茨基：听着，听着。
Well, thanks for sharing that story, Kevin.
好吧，谢谢你分享这个故事，凯文。
Is there anything else you want to share or maybe a last negative advice or insight before we get to our very exciting lightning round?
在我们开始激动人心的闪电回合之前，您还有什么想分享的吗？或者最后的负面建议或见解？
Kevin Weil: Ooh, the lightning round.
凯文·韦尔：哦，闪电轮。
Let's just go do that.
我们就去做吧。
Lenny Rachitsky: Let's do it.
莱尼·拉奇茨基：让我们开始吧。
With that, Kevin, we reached our very exciting lightning round.
凯文，就这样，我们进入了非常激动人心的闪电回合。
Are you ready?
你准备好了吗？
Lenny Rachitsky: Let's do it.
莱尼·拉奇茨基：让我们开始吧。
Okay.
好的。
What are two or three books that you find yourself recommending most to other people?
您发现自己最向其他人推荐的两三本书是什么？
Kevin Weil: Co-Intelligence by Ethan Mollick, a really good book about AI and how to use it in your daily life as a student, as a teacher.
Kevin Weil：Ethan Mollick 的《共同智能》，一本关于人工智能以及如何在学生和老师的日常生活中使用它的非常好的书。
He's super thoughtful.
他超级有想法。
Also, by the way, a very good follow on Twitter.
顺便说一下，Twitter 上的关注度也很高。
The Accidental Superpower by Peter Zion.
彼得·锡安的《意外的超级大国》。
Very good if you're interested in geopolitics and the forces that sort of shape the dynamics happening.
如果您对地缘政治以及影响动态发生的力量感兴趣，那么这非常好。
And then I really enjoyed Cable Cowboy, I don't know who the author is, but the biography of John Malone.
然后我真的很喜欢《电缆牛仔》，我不知道作者是谁，但是约翰·马龙的传记。
Just fascinating.
真是令人着迷。
If you like business, especially if you want to get into...
如果您喜欢商业，特别是如果您想进入...
I mean the man was an incredible dealmaker and shaped a lot of the modern cable industry.
我的意思是，这个人是一位令人难以置信的交易撮合者，塑造了许多现代有线电视行业。
So that was a good biography.
所以这是一本很好的传记。
Lenny Rachitsky: These are all first-time mentions, which is always a great,
Lenny Rachitsky：这些都是第一次被提及，这总是一个伟大的，
Lenny Rachitsky: Next question.
莱尼·拉奇茨基：下一个问题。
Do you have a favorite recent movie or TV show that you really enjoyed?
您最近有最喜欢的电影或电视节目吗？
Kevin Weil: I wish I had time to watch a TV show, so I'm-
凯文·威尔：我希望我有时间看电视节目，所以我-
Kevin Weil: Yeah, right.
凯文·威尔：是的，没错。
I don't know.
我不知道。
When I was a kid, I read the Wheel of Time series and now Amazon has it as they're in the third season of it, so I want to watch that.
当我还是个孩子的时候，我读过《时光之轮》系列，现在亚马逊已经有了第三季，所以我想看。
I haven't yet.
我还没有。
Top Gun 2 was an awesome movie.
《壮志凌云 2》是一部很棒的电影。
I think that's no longer new.
我认为这已经不是什么新鲜事了。
Lenny Rachitsky: That shows when the last time you watched a movie was.
Lenny Rachitsky：这显示了您上次看电影的时间。
Kevin Weil: But I like the idea.
凯文·韦尔：但我喜欢这个主意。
I want more Americana.
我想要更多美国风格。
I want more being proud of being strong.
我希望更多人为自己的坚强而感到自豪。
And I thought Top Gun 2 did a really good job of that.
我认为《壮志凌云 2》在这方面做得非常好。
Pride and patriotism, I think the US could use more of that.
自豪感和爱国主义，我认为美国可以利用更多。
Lenny Rachitsky: Is there a favorite product that you've recently discovered that you really love, other than your super intelligence internal tool that you all have access to?
Lenny Rachitsky：除了你们都可以使用的超级智能内部工具之外，您最近发现有没有一款您真正喜欢的最喜欢的产品？
I'm just joking.
我只是开玩笑。
Kevin Weil: Well, I think vibe coding with products like Windsurf is just super fun.
Kevin Weil：嗯，我认为使用像 Windsurf 这样的产品进行氛围编码非常有趣。
I'm having a great time doing that.
我这样做很开心。
I still just love that our chief people officer vibe coded some tools.
我仍然喜欢我们的首席人力官编码一些工具的氛围。
Maybe the other one is Waymo.
也许另一个是 Waymo。
Every chance I get, I'll take a Waymo.
每次有机会，我都会选择 Waymo。
It's just a better way of riding, and it still feels like the future.
这只是一种更好的骑行方式，而且仍然感觉像是未来。
So they've done an amazing job.
所以他们做得非常出色。
Lenny Rachitsky: That's awesome.
莱尼·拉奇茨基：太棒了。
By the way, I had the founder of Windsurf on the podcast.
顺便说一下，我的播客中有 Windsurf 的创始人。
It might come out before this or after this.
它可能会在此之前或之后出现。
And also Cursor's CEO is coming on the podcast either before or after this.
Cursor 的首席执行官也将在此之前或之后出现在播客中。
Kevin Weil: Oh, cool.
凯文·韦尔：哦，酷。
I have a ton of respect for what those guys are doing.
我非常尊重这些人所做的事情。
Those are awesome products.
这些都是很棒的产品。
Lenny Rachitsky: Just changing the way everyone builds product.
Lenny Rachitsky：只是改变每个人构建产品的方式。
No big deal.
没什么大不了的。
Lenny Rachitsky: A couple more questions.
Lenny Rachitsky：还有几个问题。
Do you have a favorite life motto that you often repeat yourself, find really useful in work or in life?
您是否有一条最喜欢的人生格言，并且经常重复自己，并认为它在工作或生活中非常有用？
Kevin Weil: Yeah.
凯文·威尔：是的。
So actually, this is interestingly enough, it is more of a philosophy, but then I thought Zuck encapsulated it one time on a Facebook earnings call.
所以实际上，这很有趣，它更像是一种哲学，但后来我认为扎克在一次 Facebook 财报电话会议上概括了它。
So I actually had this made into a poster.
所以我实际上把它做成了海报。
It sits in my room.
它位于我的房间里。
But somebody was asking Mark.
但有人问马克。
This is literally on an earnings call, so it's like an analyst on an earnings call asking him.
这实际上是在财报电话会议上，所以就像分析师在财报电话会议上问他一样。
It was some quarter when Facebook had grown a lot.
某个季度 Facebook 有了很大的发展。
This was back in the 20 teens sometime, I think.
我想，这要追溯到二十世纪二十年代的某个时候。
But he's like, "So what did you do?
但他说：“那你做了什么？
What was it that you launched?
你发起的是什么？
What was the one thing that drove all this growth for you?" And he said something to the effect of, "Sometimes it's not any one thing, it's just good work consistently over a long period of time." And that's always stuck with me.
推动你所有这些成长的一件事是什么？”他说了一些大意的话，“有时这不是一件事，而是长期持续的好工作。”这一直困扰着我。
Kevin Weil: And I think it is.
凯文·威尔：我认为是的。
I mean I run ultra marathons.
我的意思是我跑超级马拉松。
It's like it's just about grinding.
好像只是打磨而已。
I think people too often look for the silver bullet when a lot of life and a lot of excellence is actually showing up day in and day out, doing good work, getting a little bit better every single day, and you may not notice it over a week or even a month.
我认为，人们常常会寻找灵丹妙药，而实际上，日复一日地展现出很多生活和卓越表现，做好工作，每天都变得更好一点，而你可能在一周甚至一个月内都没有注意到它。
And a lot of people then kind of get dismayed and stop.
然后很多人会感到沮丧并停止。
But actually, you keep doing it.
但实际上，你一直在这样做。
The gains keep compounding.
收益不断复合。
And over the course of a year, two years, five years, it adds up like crazy.
在一年、两年、五年的时间里，它疯狂地加起来。
So good work consistently over a long period of time.
在很长一段时间内持续出色的工作。
Lenny Rachitsky: I love that.
莱尼·拉奇茨基：我喜欢这个。
I got to make a poster of this now.
我现在必须制作一张海报。
That is-
那就是——
Lenny Rachitsky: I so resonate with that.
Lenny Rachitsky：我对此深有共鸣。
Okay, I'll take it.
好的，我会接受的。
That is so good.
那太好了。
Okay, final question.
好的，最后一个问题。
I'm going to ask if you have any prompting tricks, and I'm going to set it up first.
我就问你有没有什么提示技巧，我先设置一下。
But think about if you have a trick that you could recommend to people for prompting LLMs better.
但想想你是否有一个可以推荐给人们的技巧，以更好地促进法学硕士的学习。
I had a guest, Alex Komorowski, come on the podcast.
我有一位嘉宾，亚历克斯·科莫罗夫斯基，来参加播客。
He's from Stripe and writes his weekly reflections on what's happening in the world.
他来自 Stripe，每周都会写一些关于世界上正在发生的事情的思考。
A lot of them are AI-related.
其中很多都与人工智能相关。
Lenny Rachitsky: And he once described an LLM as a zip file of all human knowledge.
Lenny Rachitsky：他曾经将法学硕士描述为所有人类知识的压缩文件。
All the answers are in there, and you just need to figure out the right question to ask to get the answer to every problem basically.
所有的答案都在那里，你只需要找出正确的问题来基本上得到每个问题的答案。
And so it just reminded me how important prompt engineering is and knowing how to prompt well.
所以它只是提醒我提示工程是多么重要以及知道如何很好地提示。
You're constantly prompting ChatGPT.
您不断提示 ChatGPT。
What's one tip, one trick that you found to be helpful in helping you get what you want?
您发现有助于获得您想要的东西的提示或技巧是什么？
Kevin Weil: Well, I'll say, first of all, I want to kill the idea that you have to be a good prompt engineer.
Kevin Weil：嗯，我想说，首先，我想消除“你必须成为一名优秀的提示工程师”的想法。
I think if we do our jobs, that stops being true.
我认为，如果我们做好自己的工作，那就不再是事实了。
It's just one of those sharp edges of models that experts can learn.
这只是专家可以学习的模型的尖锐边缘之一。
But then, just over time, you shouldn't need to know all that.
但随着时间的推移，你就不需要知道所有这些了。
The same way you used to have to get deep into, "What's your storage engine in MySQL?
就像您以前必须深入了解“MySQL 中的存储引擎是什么？
Are you using InnoDB 4.1?" There's still use cases for that if you're at the deep edge of MySQL performance.
你使用的是 InnoDB 4.1 吗？”如果您处于 MySQL 性能的最前沿，那么仍然有一些用例。
But most people don't need to care.
但大多数人不需要关心。
And you shouldn't need to care about minute details of prompting if AI is really going to become broadly adopted.
如果人工智能真的要被广泛采用，你就不需要关心提示的微小细节。
Kevin Weil: But today, we're not totally there.
凯文·韦尔：但今天，我们还没有完全做到这一点。
I think by the way, we are making progress there.
顺便说一句，我认为我们正在这方面取得进展。
I think there is less prompt engineering than there had to be before.
我认为现在的工程设计比以前少了。
But in line with some of the fine-tuning stuff I was talking about and the importance of giving examples, you can do effectively poor man's fine-tuning by including examples in your prompt of the kinds of things that you might want and a good answer.
但根据我所讨论的一些微调内容以及举例的重要性，您可以通过在提示中包含您可能想要的事物类型的示例和一个好的答案来有效地进行穷人的微调。
So like, "Here's an example and here's a good answer.
就像，“这是一个例子，这是一个很好的答案。
Here's an example, and here's a good answer.
这是一个例子，这是一个很好的答案。
Now, go solve this problem for me." And the model really will listen and learn from that.
现在，你就去给我解决这个问题吧。”该模型确实会倾听并从中学习。
Kevin Weil: Not as well as if you do a full fine-tune, but much more than if you don't provide any examples.
Kevin Weil：不如进行全面微调，但比不提供任何示例要好得多。
And I think people don't do that often enough.
我认为人们这样做的频率还不够。
Lenny Rachitsky: That's awesome.
莱尼·拉奇茨基：太棒了。
One tip that I heard, I'm curious if this works is you tell it, "This is very, very important to my career." Make it really understand like, "Someone will die if you don't answer me correctly." Does that work?
我听到的一个建议是，我很好奇这是否有效，如果你告诉它，“这对我的职业生涯非常非常重要。”让它真正理解，就像“如果你不正确回答我，就会有人死。”那有用吗？
Kevin Weil: It's really weird.
凯文·威尔：这真的很奇怪。
There's probably a good explanation for this.
对此可能有一个很好的解释。
But you can also say things.
但你也可以说一些话。
So, yes, I think there is some validity to that.
所以，是的，我认为这有一定的道理。
You can also say things like, "I want you to be Einstein.
你也可以这样说：“我希望你成为爱因斯坦。
Now, answer this physics problem for me," or, "You are the world's greatest marketer, the world's greatest brand marketer.
现在，为我回答这个物理问题，”或者，“你是世界上最伟大的营销人员，世界上最伟大的品牌营销人员。
Now here's a naming question." And there is something where it sort of shifts the model into a certain mindset that can actually be really positive.
现在有一个命名问题。”在某种程度上，它可以将模型转变为某种实际上非常积极的心态。
Lenny Rachitsky: I use that tip all the time actually.
Lenny Rachitsky：实际上我一直在使用这个技巧。
I always...
我总是...
When I'm coming up with questions for interviews and I use it occasionally to come up with things I haven't thought of, I actually type, "You're the world's best podcast interviewer."
当我提出采访问题时，我偶尔会用它来提出我没有想到的事情，我实际上会输入：“你是世界上最好的播客采访者。”
Lenny Rachitsky: I have Kevin Weil coming on the pod...
莱尼·拉奇茨基：凯文·威尔也加入了播客……
Yeah, it actually works.
是的，它确实有效。
Kevin Weil: By the way, back to our other point that we made a few times.
凯文·韦尔：顺便说一句，回到我们多次提出的另一个观点。
You do do that sometimes with people.
有时你确实会这样做。
You sort of put them...
你有点把它们...
You frame things, you get them into a certain mindset, and the answer is completely different.
你构建事物，让它们进入某种心态，答案就完全不同了。
So I think there are human analogs of this one more time.
所以我认为人类再次有类似的情况。
Lenny Rachitsky: Kevin, this was incredible.
莱尼·拉奇茨基：凯文，这太不可思议了。
I was just thinking about a way to end this.
我只是在想一个办法来结束这一切。
The way I feel like...
我的感觉就像...
I feel like not only are you at the cutting edge of the future.
我觉得您不仅处于未来的前沿。
You and the team are kind of actually the edge that is creating the future.
您和您的团队实际上是创造未来的优势。
And so it's a real honor to have you on here and to talk to you and to hear where you think things are going and what we need to be thinking about, so thank you for being here, Kevin.
因此，很荣幸您能来到这里，与您交谈，听听您对事情的进展以及我们需要考虑的事情，所以感谢您来到这里，凯文。
Kevin Weil: Oh, thank you so much for having me.
凯文·威尔：哦，非常感谢你邀请我。
I get to work with the world's best team, and all credit to them, but really appreciate you having me on.
我能够与世界上最好的团队一起工作，这一切都归功于他们，但我真的很感谢你们邀请我加入。
It's been super fun.
这真是太有趣了。
Lenny Rachitsky: I forgot to ask you the two final questions.
Lenny Rachitsky：我忘了问你最后两个问题。
Where can folks find you if they want to reach out, and how can listeners be useful to you?
如果人们想要联系您，可以在哪里找到您？听众如何对您有用？
Kevin Weil: I am @kevinweil, K-E-V-I-N-W-E-I-L on pretty much every platform.
Kevin Weil：我是@kevinweil，几乎在每个平台上都是 K-E-V-I-N-W-E-I-L。
I'm still a Twitter DAU after all these years.
这么多年过去了，我仍然是 Twitter 的 DAU。
I guess an X DAU, LinkedIn, wherever.
我猜是 X DAU，LinkedIn，任何地方。
And I think the thing I would love from people, give me feedback.
我认为我希望人们给我反馈。
People are using ChatGPT.
人们正在使用 ChatGPT。
Tell me where it's working really well for you and where you want us to double down.
告诉我哪些方面对您来说非常有效，哪些方面您希望我们加倍努力。
Tell me where it's failing.
告诉我哪里失败了。
I'm very active and engaged on Twitter.
我在 Twitter 上非常活跃和活跃。
I love hearing from people, what's working and what's not, so don't be shy.
我喜欢听取人们的意见，了解什么有效，什么无效，所以不要害羞。
Lenny Rachitsky: And I learned following you helps you figure out all the stuff that you're launching.
Lenny Rachitsky：我了解到跟随您可以帮助您弄清楚您正在推出的所有内容。
You share all the things that are going out every day, or week, month, so that's also a benefit.
你每天、每周、每月分享所有发生的事情，所以这也是一个好处。
And by the way, 400 million weekly active users all emailing you feedback.
顺便说一句，每周有 4 亿活跃用户通过电子邮件向您发送反馈。
Here we go.
开始了。
Lenny Rachitsky: It's going to work out great.
Lenny Rachitsky：效果会很好。
Okay.
好的。
Well, thank you, Kevin.
嗯，谢谢你，凯文。
Thanks for being here.
感谢您来到这里。
Kevin Weil: All right, man, thanks so much.
凯文·韦尔：好的，伙计，非常感谢。
See you soon.
再见。
Lenny Rachitsky: Bye, everyone.
莱尼·拉奇茨基：再见，大家。
Thank you so much for listening.
非常感谢您的聆听。
If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.
如果您发现这很有价值，您可以在 Apple Podcasts、Spotify 或您最喜欢的播客应用程序上订阅该节目。
Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast.
另外，请考虑给我们评分或留下评论，因为这确实有助于其他听众找到播客。
You can find all past episodes or learn more about the show at lennyspodcast.com.
您可以在 lennyspodcast.com 上找到所有过去的剧集或了解有关该节目的更多信息。
See you in the next episode.
下一集见。