达里奥·阿莫代伊：Anthropic CEO 谈 Claude、AGI 与 AI 及人类的未来

If you extrapolate the curves that we’ve had so far, right?

如果你推断出我们迄今为止的曲线，对吗？

00:00 If you say, “Well, I don’t know, we’re starting to get to PhD level, and last year we were at undergraduate level, and the year before we were at the level of a high school student,” again, you can quibble with what tasks and for what.

如果你说，“嗯，我不知道，我们开始达到博士学位水平，去年我们处于本科水平，前一年我们处于高中生水平，”你可以再次争论什么任务和目的。

“We’re still missing modalities, but those are being added,” like computer use was added, like image generation has been added.

“我们仍然缺少模式，但这些正在被添加，”就像添加了计算机使用一样，就像添加了图像生成一样。

If you just kind of eyeball the rate at which these capabilities are increasing, it does make you think that we’ll get there by 2026 or 2027.

如果您关注这些能力增长的速度，您确实会认为我们将在 2026 年或 2027 年实现这一目标。

00:31 I think there are still worlds where it doesn’t happen in 100 years.

我认为仍然有一些世界 100 年后不会发生这种情况。

The number of those worlds is rapidly decreasing.

这些世界的数量正在迅速减少。

We are rapidly running out of truly convincing blockers, truly compelling reasons why this will not happen in the next few years.

我们正在迅速耗尽真正令人信服的阻碍因素，以及在未来几年内不会发生这种情况的真正令人信服的理由。

The scale-up is very quick.

规模扩大的速度非常快。

We do this today, we make a model, and then we deploy thousands, maybe tens of thousands of instances of it.

我们今天这样做，创建一个模型，然后部署数千个甚至数万个它的实例。

I think by the time, certainly within two to three years, whether we have these super powerful AIs or not, clusters are going to get to the size where you’ll be able to deploy millions of these.

我认为，到那时，当然是在两到三年内，无论我们是否拥有这些超级强大的人工智能，集群都将达到能够部署数百万个的规模。

01:03 I am optimistic about meaning.

我对意义持乐观态度。

I worry about economics and the concentration of power.

我担心经济和权力集中。

That’s actually what I worry about more, the abuse of power.

其实我更担心的是滥用权力。

And AI increases the amount of power in the world.

人工智能增加了世界的力量。

01:14 And if you concentrate that power and abuse that power, it can do immeasurable damage.

如果你集中这种力量并滥用这种力量，它会造成不可估量的损害。

Yes, it’s very frightening.

是的，这非常可怕。

01:22 It’s very frightening.

这非常可怕。

The following is a conversation with Dario Amodei, CEO of Anthropic, the company that created Claude, that is currently and often at the top of most LLM benchmark leader boards.

以下是与 Anthropic 首席执行官达里奥·阿莫迪 (Dario Amodei) 的对话，该公司创建了 Claude，该公司目前经常位居大多数 LLM 基准排行榜的榜首。

01:27 On top of that, Dario and the Anthropic team have been outspoken advocates for taking the topic of AI safety very seriously.

最重要的是，Dario 和 Anthropic 团队一直直言不讳地倡导非常严肃地对待人工智能安全话题。

And they have continued to publish a lot of fascinating AI on this and other topics.

他们继续就这个主题和其他主题发表许多引人入胜的人工智能文章。

01:55 I’m also joined afterwards by two other brilliant people from Anthropic.

随后，来自 Anthropic 的另外两位杰出人士也加入了我的行列。

First Amanda Askell, who is a researcher working on alignment and fine-tuning of Claude, including the design of Claude’s character and personality.

首先是阿曼达·阿斯克尔（Amanda Askell），她是一名研究员，致力于克劳德的对齐和微调，包括克劳德的角色和个性的设计。

A few folks told me she has probably talked with Claude more than any human at Anthropic.

有几个人告诉我，她与克劳德的交谈可能比 Anthropic 的任何人都多。

So she was definitely a fascinating person to talk to about prompt engineering and practical advice on how to get the best out of Claude.

因此，她绝对是一个令人着迷的人，可以与她谈论及时的工程设计以及如何充分发挥克劳德的实用建议。

02:27 After that, Chris Olah stopped by for a chat.

之后，克里斯·奥拉 (Chris Olah) 过来聊天。

He’s one of the pioneers of the field of mechanistic interpretability, which is an exciting set of efforts that aims to reverse engineering neural networks, to figure out what’s going on inside, inferring behaviors from neural activation patterns inside the network.

他是机械可解释性领域的先驱之一，该领域是一系列令人兴奋的工作，旨在对神经网络进行逆向工程，弄清楚内部发生了什么，从网络内部的神经激活模式推断行为。

This is a very promising approach for keeping future super-intelligent AI systems safe.

这是保证未来超级智能人工智能系统安全的一种非常有前景的方法。

For example, by detecting from the activations when the model is trying to deceive the human it is talking to.

例如，通过从激活中检测模型何时试图欺骗与之交谈的人类。

03:03 This is the Lex Fridman podcast.

这是莱克斯·弗里德曼播客。

To support it, please check out our sponsors in the description.

为了支持它，请在说明中查看我们的赞助商。

And now, dear friends, here’s Dario Amodei.

现在，亲爱的朋友们，这是达里奥·阿莫代。

03:14 Let’s start with a big idea of scaling laws and the scaling hypothesis.

让我们从缩放定律和缩放假设的大概念开始。

What is it?

它是什么？

What is its history, and where do we stand today?

它的历史是什么？我们今天处于什么位置？

So I can only describe it as it relates to my own experience, but I’ve been in the AI field for about 10 years and it was something I noticed very early on.

所以我只能根据我自己的经历来描述它，但我在人工智能领域已经有大约10年了，这是我很早就注意到的事情。

03:22 So I first joined the AI world when I was working at Baidu with Andrew Ng in late 2014, which is almost exactly 10 years ago now.

所以我第一次进入人工智能世界是在 2014 年底，当时我在百度与吴恩达一起工作，现在已经快整整 10 年前了。

And the first thing we worked on, was speech recognition systems.

我们做的第一件事是语音识别系统。

And in those days I think deep learning was a new thing.

在那些日子里，我认为深度学习是一个新事物。

It had made lots of progress, but everyone was always saying, “We don’t have the algorithms we need to succeed.

它已经取得了很大进展，但每个人总是说：“我们没有成功所需的算法。

We are only matching a tiny fraction.

我们只匹配一小部分。

There’s so much we need to discover algorithmically.

我们需要通过算法来发现很多东西。

We haven’t found the picture of how to match the human brain.”

我们还没有找到如何匹配人脑的图片。”

04:05 And in some ways it was fortunate, you can have almost beginner’s luck.

从某些方面来说，这是幸运的，你几乎可以拥有初学者的运气。

I was like a newcomer to the field.

我就像是这个领域的新人。

And I looked at the neural net that we were using for speech, the recurrent neural networks, and I said, “I don’t know, what if you make them bigger and give them more layers?

我看了看我们用于语音的神经网络，即循环神经网络，我说，“我不知道，如果你让它们变得更大并给它们更多的层会怎么样？

And what if you scale up the data along with this?” I just saw these as independent dials that you could turn.

如果你同时扩大数据规模会怎么样？”我只是将它们视为可以转动的独立转盘。

And I noticed that the models started to do better and better as you gave them more data, as you made the models larger, as you trained them for longer.

我注意到，当你给模型提供更多数据、模型变得更大、训练时间更长时，模型开始表现得越来越好。

And I didn’t measure things precisely in those days, but along with colleagues, we very much got the informal sense that the more data and the more compute and the more training you put into these models, the better they perform.

那时我并没有精确地测量事物，但与同事们一起，我们非常有一种非正式的感觉，即你在这些模型中投入的数据越多、计算越多、训练越多，它们的性能就越好。

04:51 And so initially my thinking was, “Hey, maybe that is just true for speech recognition systems.

所以最初我的想法是，“嘿，也许这对于语音识别系统来说是正确的。

Maybe that’s just one particular quirk, one particular area.” I think it wasn’t until 2017 when I first saw the results from GPT-1 that it clicked for me that language is probably the area in which we can do this.

也许这只是一个特殊的怪癖，一个特定的领域。”我认为直到 2017 年，当我第一次看到 GPT-1 的结果时，我才意识到语言可能是我们可以做到这一点的领域。

We can get trillions of words of language data, we can train on them.

我们可以获得数万亿个单词的语言数据，我们可以对它们进行训练。

And the models we were trained in those days were tiny.

那时我们训练的模型很小。

You could train them on one to eight GPUs, whereas now we train jobs on tens of thousands, soon going to hundreds of thousands of GPUs.

您可以在一到八个 GPU 上训练它们，而现在我们在数万个 GPU 上训练作业，很快就会达到数十万个 GPU。

05:28 And so when I saw those two things together, and there were a few people like Ilya Sudskever who you’ve interviewed, who had somewhat similar views.

所以当我把这两件事放在一起看到时，你采访过的一些像 Ilya Sudskever 这样的人也有一些相似的观点。

He might’ve been the first one, although I think a few people came to similar views around the same time, right?

他可能是第一个，尽管我认为一些人大约在同一时间提出了类似的观点，对吗？

There was Rich Sutton’s bitter lesson, Gwern wrote about the scaling hypothesis.

格温（Gwern）写了有关缩放假设的文章，其中有里奇·萨顿（Rich Sutton）的惨痛教训。

But I think somewhere between 2014 and 2017 was when it really clicked for me, when I really got conviction that, “Hey, we’re going to be able to these incredibly wide cognitive tasks if we just scale up the models.”

但我认为在 2014 年到 2017 年之间的某个时候，我真的很感动，当时我真正确信，“嘿，如果我们扩大模型规模，我们将能够完成这些令人难以置信的广泛认知任务。”

06:03 And at every stage of scaling, there are always arguments.

在扩展的每个阶段，总是存在争论。

And when I first heard them honestly, I thought, “Probably I’m the one who’s wrong and all these experts in the field are right.

当我第一次诚实地听到他们时，我想：“可能我是错的，而该领域的所有这些专家都是对的。

They know the situation better than I do, right?” There’s the Chomsky argument about, “You can get syntactics but you can’t get semantics.” There was this idea, “Oh, you can make a sentence make sense, but you can’t make a paragraph make sense.” The latest one we have today is, “We’re going to run out of data, or the data isn’t high quality enough or models can’t reason.”

他们比我更了解情况，对吧？”乔姆斯基的论点是：“你可以获得句法，但无法获得语义。”有这样的想法，“哦，你可以让一个句子有意义，但你不能让一个段落有意义。”我们今天得到的最新消息是，“我们将耗尽数据，或者数据质量不够高，或者模型无法推理。”

06:34 And each time, every time, we manage to either find a way around or scaling just is the way around.

每一次，每一次，我们要么设法找到解决办法，要么扩大规模就是解决办法。

Sometimes it’s one, sometimes it’s the other.

有时是一个，有时是另一个。

And so I’m now at this point, I still think it’s always quite uncertain.

所以现在我仍然认为它总是相当不确定。

We have nothing but inductive inference to tell us that the next two years are going to be like the last 10 years.

除了归纳推理之外，我们什么也没有告诉我们，未来两年将会像过去十年一样。

But I’ve seen the movie enough times, I’ve seen the story happen for enough times to really believe that probably the scaling is going to continue, and that there’s some magic to it that we haven’t really explained on a theoretical basis yet.

但我已经看过这部电影足够多的次数了，我看过这个故事发生的次数足够多了，我真的相信缩放比例可能会继续下去，而且它有一些魔力，我们还没有在理论基础上真正解释过。

And of course the scaling here is bigger networks, bigger data, bigger compute?

当然，这里的扩展是指更大的网络、更大的数据、更大的计算？

Yes.

是的。

All of those?

所有这些？

In particular, linear scaling up of bigger networks, bigger training times and more and more data.

特别是更大网络的线性扩展、更长的训练时间和越来越多的数据。

07:17 So all of these things, almost like a chemical reaction, you have three ingredients in the chemical reaction and you need to linearly scale up the three ingredients.

所以所有这些事情，几乎就像一个化学反应，化学反应中有三种成分，你需要线性放大这三种成分。

If you scale up one, not the others, you run out of the other reagents and the reaction stops.

如果扩大其中一种试剂的规模，而不扩大其他试剂的规模，则会耗尽其他试剂，反应就会停止。

But if you scale up everything in series, then the reaction can proceed.

但如果你按顺序放大所有东西，那么反应就可以继续进行。

And of course now that you have this kind of empirical science/art, you can apply it to other more nuanced things like scaling laws applied to interpretability or scaling laws applied to post-training.

当然，现在你有了这种经验科学/艺术，你可以将其应用于其他更微妙的事情，例如应用于可解释性的缩放定律或应用于训练后的缩放定律。

07:45 Or just seeing how does this thing scale.

或者只是看看这个东西如何扩展。

But the big scaling law, I guess the underlying scaling hypothesis has to do with big networks, big data leads to intelligence?

但是大尺度定律，我猜潜在的尺度假设与大网络有关，大数据导致智能？

Yeah, we’ve documented scaling laws in lots of domains other than language.

是的，我们已经记录了除语言之外的许多领域的缩放定律。

08:09 So initially the paper we did that first showed it, was in early 2020, where we first showed it for language.

因此，我们最初发表的论文是在 2020 年初首次展示了它，当时我们首次展示了它的语言功能。

There was then some work late in 2020 where we showed the same thing for other modalities like images, video, text to image, image to text, math.

2020 年末，我们进行了一些工作，针对图像、视频、文本到图像、图像到文本、数学等其他模式展示了同样的结果。

They all had the same pattern.

他们都有相同的模式。

And you’re right, now there are other stages like post-training or there are new types of reasoning models.

你是对的，现在还有其他阶段，例如训练后或新型推理模型。

And in all of those cases that we’ve measured, we see similar types of scaling laws.

在我们测量的所有这些情况下，我们看到了类似类型的缩放定律。

A bit of a philosophical question, but what’s your intuition about why bigger is better in terms of network size and data size?

这是一个有点哲学问题，但是您对为什么网络规模和数据规模越大越好的直觉是什么？

08:48 Why does it lead to more intelligent models?

为什么它会带来更智能的模型？

So in my previous career as a biophysicist…

所以在我之前作为生物物理学家的职业生涯中......

09:00 So I did a physics undergrad and then biophysics in grad school.

所以我读了物理学本科，然后在研究生院读了生物物理学。

So I think back to what I know as a physicist, which is actually much less than what some of my colleagues at Anthropic have in terms of expertise in physics.

所以我回想一下我作为一名物理学家所了解的，实际上比我在 Anthropic 的一些同事在物理学方面的专业知识要少得多。

There’s this concept called the one over F noise and one over X distributions, where often, just like if you add up a bunch of natural processes, you get a Gaussian, if you add up a bunch of differently-distributed natural processes…

有一个概念称为 one over F 噪声和 one over X 分布，通常，就像将一堆自然过程加起来一样，如果将一堆不同分布的自然过程加起来，就会得到高斯分布……

If you take a probe and hook it up to a resistor, the distribution of the thermal noise in the resistor goes as one over the frequency.

如果您将探头连接到电阻器上，则电阻器中的热噪声分布将随频率变化。

It’s some kind of natural convergent distribution.

这是某种自然收敛分布。

09:50 And I think what it amounts to, is that if you look at a lot of things that are produced by some natural process that has a lot of different scales, not a Gaussian, which is kind of narrowly distributed, but if I look at large and small fluctuations that lead to electrical noise, they have this decaying one over X distribution.

我认为它的意思是，如果你观察一些自然过程产生的很多东西，这些过程有很多不同的尺度，而不是高斯分布，它的分布很窄，但如果我观察导致电噪声的大大小小的波动，它们就会具有这种衰减的 X 分布。

And so now I think of patterns in the physical world or in language.

所以现在我想到了物理世界或语言中的模式。

If I think about the patterns in language, there are some really simple patterns, some words are much more common than others, like the.

如果我考虑一下语言中的模式，有一些非常简单的模式，有些单词比其他单词更常见，比如 the。

Then there’s basic noun-verb structure.

然后是基本的名词-动词结构。

Then there’s the fact that nouns and verbs have to agree, they have to coordinate.

事实上，名词和动词必须一致，必须协调。

And there’s the higher-level sentence structure.

还有更高级的句子结构。

Then there’s the thematic structure of paragraphs.

然后是段落的主题结构。

And so the fact that there’s this regressing structure, you can imagine that as you make the networks larger, first they capture the really simple correlations, the really simple patterns, and there’s this long tail of other patterns.

因此，事实上存在这种回归结构，您可以想象，当您使网络变得更大时，它们首先捕获非常简单的相关性、非常简单的模式，并且还有其他模式的长尾。

10:49 And if that long tail of other patterns is really smooth like it is with the one over F noise in physical processes like resistors, then you can imagine as you make the network larger, it’s kind of capturing more and more of that distribution.

如果其他模式的长尾确实很平滑，就像电阻器等物理过程中的 F 噪声一样，那么您可以想象，当您使网络更大时，它会捕获越来越多的分布。

And so that smoothness gets reflected in how well the models are at predicting and how well they perform.

因此，平滑度反映在模型的预测能力和执行能力上。

11:10 Language is an evolved process.

语言是一个进化的过程。

We’ve developed language, we have common words and less common words.

我们已经发展了语言，我们有常见的单词和不常见的单词。

We have common expressions and less common expressions.

我们有常见的表达方式和不太常见的表达方式。

We have ideas, cliches, that are expressed frequently, and we have novel ideas.

我们有经常表达的想法、陈词滥调，也有新颖的想法。

And that process has developed, has evolved with humans over millions of years.

这个过程已经随着人类的发展、进化了数百万年。

And so the guess, and this is pure speculation, would be that there’s some kind of long tail distribution of the distribution of these ideas.

所以猜测，这纯粹是猜测，这些想法的分布存在某种长尾分布。

So there’s the long tail, but also there’s the height of the hierarchy of concepts that you’re building up.

所以有长尾，但也有你正在建立的概念层次结构的高度。

11:41 So the bigger the network, presumably you have a higher capacity to-

因此，网络越大，想必您的能力就越高-

Exactly.

正是如此。

11:50 If you have a small network, you only get the common stuff.

如果你的网络很小，你只能得到常见的东西。

If I take a tiny neural network, it’s very good at understanding that a sentence has to have verb, adjective, noun, but it’s terrible at deciding what those verb adjective and noun should be and whether they should make sense.

如果我采用一个小型神经网络，它非常擅长理解一个句子必须有动词、形容词、名词，但在决定这些动词、形容词和名词应该是什么以及它们是否有意义时却很糟糕。

If I make it just a little bigger, it gets good at that, then suddenly it’s good at the sentences, but it’s not good at the paragraphs.

如果我把它做得大一点，它就会变得擅长，然后突然它擅长句子，但不擅长段落。

And so these rarer and more complex patterns get picked up as I add more capacity to the network.

因此，当我向网络添加更多容量时，就会发现这些更罕见、更复杂的模式。

Well, the natural question then is what’s the ceiling of this?

那么，自然的问题是这个上限是多少？

Yeah.

是的。

How complicated and complex is the real world?

现实世界到底有多复杂、复杂？

12:24 How much is the stuff is there to learn?

有多少东西可以学习？

I don’t think any of us knows the answer to that question.

我认为我们中没有人知道这个问题的答案。

12:30 My strong instinct would be that there’s no ceiling below the level of humans.

我强烈的直觉是，人类的水平没有上限。

We humans are able to understand these various patterns.

我们人类能够理解这些不同的模式。

And so that makes me think that if we continue to scale up these models to kind of develop new methods for training them and scaling them up, that will at least get to the level that we’ve gotten to with humans.

因此，这让我认为，如果我们继续扩大这些模型的规模，开发新的方法来训练它们并扩大规模，那么至少会达到我们人类所达到的水平。

There’s then a question of how much more is it possible to understand than humans do?

那么问题来了，它的理解能力比人类多多少？

How much is it possible to be smarter and more perceptive than humans?

比人类更聪明、更有洞察力的可能性有多大？

I would guess the answer has got to be domain-dependent.

我猜答案必须与领域相关。

13:09 If I look at an area like biology, and I wrote this essay, Machines of Loving Grace, it seems to me that humans are struggling to understand the complexity of biology.

如果我看看生物学这样的领域，我写了这篇文章《爱的机器》，在我看来，人类正在努力理解生物学的复杂性。

If you go to Stanford or to Harvard or to Berkeley, you have whole departments of folks trying to study the immune system or metabolic pathways, and each person understands only a tiny bit, a part of it, specializes.

如果你去斯坦福大学、哈佛或伯克利，你会有整个部门的人试图研究免疫系统或代谢途径，而每个人只了解其中的一小部分，专业知识。

And they’re struggling to combine their knowledge with that of other humans.

他们正在努力将自己的知识与其他人类的知识结合起来。

And so I have an instinct that there’s a lot of room at the top for AIs to get smarter.

所以我有一种直觉，人工智能在高层有很大的空间可以变得更聪明。

13:46 If I think of something like materials in the physical world, or addressing conflicts between humans or something like that, I mean it may be there’s only some of these problems are not intractable, but much harder.

如果我想到物理世界中的材料之类的问题，或者解决人类之间的冲突之类的问题，我的意思是，可能只有其中一些问题并不棘手，而是更加困难。

And it may be that there’s only so well you can do at some of these things.

也许你在其中一些事情上能做得很好。

Just like with speech recognition, there’s only so clear I can hear your speech.

就像语音识别一样，我只能清晰地听到你的讲话。

So I think in some areas there may be ceilings that are very close to what humans have done.

所以我认为在某些领域可能存在非常接近人类所做的事情的上限。

In other areas, those ceilings may be very far away.

在其他地区，这些天花板可能非常遥远。

I think we’ll only find out when we build these systems.

我想只有当我们构建这些系统时我们才会知道。

It’s very hard to know in advance.

提前知道是非常困难的。

We can speculate, but we can’t be sure.

我们可以推测，但不能确定。

And in some domains, the ceiling might have to do with human bureaucracies and things like this, as you write about.

正如您所写，在某些领域，上限可能与人类官僚机构和类似的事情有关。

Yes.

是的。

So humans fundamentally has to be part of the loop.

因此，人类从根本上必须成为循环的一部分。

14:31 That’s the cause of the ceiling, not maybe the limits of the intelligence.

这就是上限的原因，而不是智力的极限。

Yeah, I think in many cases, in theory, technology could change very fast.

是的，我认为在很多情况下，理论上，技术可能变化得非常快。

14:38 For example, all the things that we might invent with respect to biology, but remember, there’s a clinical trial system that we have to go through to actually administer these things to humans.

例如，我们可能发明与生物学有关的所有东西，但请记住，我们必须通过一个临床试验系统才能将这些东西实际应用于人类。

I think that’s a mixture of things that are unnecessary in bureaucratic and things that kind of protect the integrity of society.

我认为这是官僚主义中不必要的事情和保护社会诚信的事情的混合体。

And the whole challenge is that it’s hard to tell what’s going on.

整个挑战在于很难判断发生了什么。

It’s hard to tell which is which.

很难区分哪个是哪个。

15:11 I think in terms of drug development, my view is that we’re too slow and we’re too conservative.

我认为在药物开发方面，我的观点是我们太慢而且太保守。

But certainly if you get these things wrong, it’s possible to risk people’s lives by being too reckless.

但当然，如果你把这些事情搞错了，就可能因为过于鲁莽而冒着生命危险。

And so at least some of these human institutions are in fact protecting people.

因此，至少其中一些人类机构实际上正在保护人民。

So it’s all about finding the balance.

所以一切都是为了找到平衡。

I strongly suspect that balance is kind of more on the side of wishing to make things happen faster, but there is a balance.

我强烈怀疑平衡更多的是希望让事情发生得更快，但平衡是存在的。

If we do hit a limit, if we do hit a slowdown in the scaling laws, what do you think would be the reason?

如果我们确实达到了极限，如果我们确实达到了缩放定律的放缓，你认为原因是什么？

15:39 Is it compute-limited, data-limited?

是计算受限、数据受限吗？

Is it something else?

难道是别的什么？

Idea limited?

想法有限？

So a few things, now we’re talking about hitting the limit before we get to the level of humans and the skill of humans.

有几件事，现在我们谈论的是在达到人类水平和人类技能之前达到极限。

15:51 So I think one that’s popular today, and I think could be a limit that we run into, like most of the limits, I would bet against it, but it’s definitely possible, is we simply run out of data.

所以我认为今天很流行，我认为这可能是我们遇到的一个限制，就像大多数限制一样，我打赌反对它，但这绝对是可能的，因为我们只是耗尽了数据。

There’s only so much data on the internet, and there’s issues with the quality of the data.

互联网上的数据有限，而且数据质量存在问题。

You can get hundreds of trillions of words on the internet, but a lot of it is repetitive or it’s search engine optimization drivel, or maybe in the future it’ll even be text generated by AIs itself.

你可以在互联网上获得数百万亿个单词，但其中很多是重复的，或者是搜索引擎优化的胡言乱语，或者将来甚至可能是人工智能本身生成的文本。

And so I think there are limits to what can be produced in this way.

所以我认为以这种方式生产的东西是有限的。

16:34 That said, we, and I would guess other companies, are working on ways to make data synthetic, where you can use the model to generate more data of the type that you have already, or even generate data from scratch.

也就是说，我们，我猜其他公司正在研究合成数据的方法，您可以使用该模型生成更多已有类型的数据，甚至从头开始生成数据。

If you think about what was done with DeepMind’s AlphaGo Zero, they managed to get a bot all the way from no ability to play Go whatsoever to above human level, just by playing against itself.

如果你想一想 DeepMind 的 AlphaGo Zero 所做的事情，你就会发现，他们仅仅通过与自己对弈，就成功地让机器人从不会下围棋的能力一路提升到高于人类水平。

There was no example data from humans required in the AlphaGo Zero version of it.

AlphaGo Zero 版本中不需要来自人类的示例数据。

17:07 The other direction of course, is these reasoning models that do chain of thought and stop to think and reflect on their own thinking.

当然另一个方向，就是这些做思维链条、停下来思考、反思自己的思维的推理模型。

In a way that’s another kind of synthetic data coupled with reinforcement learning.

在某种程度上，这是另一种与强化学习相结合的合成数据。

So my guess is with one of those methods, we’ll get around the data limitation or there may be other sources of data that are available.

所以我的猜测是，使用其中一种方法，我们将绕过数据限制，或者可能有其他可用的数据源。

We could just observe that, even if there’s no problem with data, as we start to scale models up, they just stopped getting better.

我们可以观察到，即使数据没有问题，当我们开始扩大模型规模时，它们就不再变得更好了。

It seemed to be a reliable observation that they’ve gotten better, that could just stop at some point for a reason we don’t understand.

这似乎是一个可靠的观察结果，他们已经变得更好了，但这种情况可能会因为我们不明白的原因而在某个时候停止。

17:43 The answer could be that we need to invent some new architecture.

答案可能是我们需要发明一些新的架构。

There have been problems in the past with say, numerical stability of models where it looked like things were leveling off, but actually when we found the right unblocker, they didn’t end up doing so.

过去存在一些问题，比如模型的数值稳定性，看起来一切都趋于平稳，但实际上，当我们找到合适的解锁器时，他们最终并没有这样做。

So perhaps there’s some new optimization method or some new technique we need to unblock things.

因此，也许我们需要一些新的优化方法或新技术来解锁。

I’ve seen no evidence of that so far, but if things were to slow down, that perhaps could be one reason.

到目前为止，我还没有看到任何证据，但如果事情放缓，这可能是原因之一。

What about the limits of compute, meaning the expensive nature of building bigger and bigger data centers?

计算的局限性如何，意味着建造越来越大的数据中心的成本高昂？

So right now, I think most of the frontier model companies, I would guess, are operating in roughly 1 billion scale, plus or minus a factor of three.

所以现在，我认为大多数前沿模型公司的运营规模约为 10 亿，正负三倍。

18:23 Those are the models that exist now or are being trained now.

这些是现在存在或正在训练的模型。

I think next year we’re going to go to a few billion, and then 2026, we may go to above 10 billion.

我认为明年我们会达到几十亿，然后到2026年，我们可能会超过100亿。

And probably by 2027, their ambitions to build hundred billion dollar clusters.

或许到 2027 年，他们将雄心勃勃地打造价值千亿美元的集群。

And I think all of that actually will happen.

我认为这一切实际上都会发生。

There’s a lot of determination to build the compute, to do it within this country, and I would guess that it actually does happen.

人们决心在这个国家内构建计算，我猜这确实会发生。

19:02 Now, if we get to a hundred billion, that’s still not enough compute, that’s still not enough scale, then either we need even more scale, or we need to develop some way of doing it more efficiently of shifting the curve.

现在，如果我们达到了 1000 亿，计算量仍然不够，规模仍然不够，那么我们要么需要更大的规模，要么需要开发某种方法来更有效地改变曲线。

I think between all of these, one of the reasons I’m bullish about powerful AI happening so fast, is just that if you extrapolate the next few points on the curve, we’re very quickly getting towards human level ability.

我认为在所有这些之中，我看好强大的人工智能如此迅速地发生的原因之一就是，如果你推断曲线上接下来的几个点，我们很快就会达到人类水平的能力。

19:28 Some of the new models that we developed, some reasoning models that have come from other companies, they’re starting to get to what I would call the PhD or professional level.

我们开发的一些新模型，一些来自其他公司的推理模型，它们开始达到我所说的博士或专业水平。

If you look at their coding ability, the latest model we released, Sonnet 3.5, the new or updated version, it gets something like 50% on SWE-bench.

如果你看看他们的编码能力，我们发布的最新模型，Sonnet 3.5，新的或更新的版本，它在 SWE-bench 上的得分约为 50%。

And SWE-bench is an example of a bunch of professional real-world software engineering tasks.

SWE-bench 是一系列专业的现实世界软件工程任务的示例。

At the beginning of the year, I think the state of the art was 3 or 4%.

今年年初，我认为最先进的水平是 3% 或 4%。

So in 10 months we’ve gone from 3% to 50% on this task.

因此，在 10 个月内，我们完成这项任务的比例从 3% 增加到了 50%。

And I think in another year we’ll probably be at 90%.

我认为再过一年我们可能会达到 90%。

I mean, I don’t know, but might even be less than that.

我的意思是，我不知道，但可能比这个还要少。

20:11 We’ve seen similar things in graduate-level math, physics, and biology from models like OpenAi’s o1.

我们在研究生水平的数学、物理和生物学领域从 OpenAi 的 o1 等模型中看到了类似的情况。

So if we just continue to extrapolate this in terms of skill that we have, I think if we extrapolate the straight curve, within a few years, we will get to these models being above the highest professional level in terms of humans.

因此，如果我们继续根据我们所拥有的技能来推断，我认为如果我们推断出直线，几年之内，我们将获得高于人类最高专业水平的这些模型。

Now, will that curve continue?

现在，这条曲线会继续下去吗？

You’ve pointed to, and I’ve pointed to a lot of possible reasons why that might not happen.

你已经指出了，我也指出了许多可能不会发生这种情况的原因。

But if the extrapolation curve continues, that is the trajectory we’re on.

但如果外推曲线继续下去，那就是我们所处的轨迹。

So Anthropic has several competitors.

因此，Anthropic 有几个竞争对手。

20:46 It’d be interesting to get your sort of view of it all.

了解你对这一切的看法会很有趣。

OpenAI, Google, XAI, Meta.

OpenAI、谷歌、XAI、Meta。

What does it take to win in the broad sense of win in this space?

在这个领域，要取得广义的胜利需要什么？

Yeah, so I want to separate out a couple things, right?

是的，所以我想把几件事分开，对吗？

20:58 Anthropic’s mission is to kind of try to make this all go well.

Anthropic 的使命是努力让这一切顺利进行。

And we have a theory of change called Race to the Top.

我们有一个名为“力争上游”的变革理论。

Race to the Top is about trying to push the other players to do the right thing by setting an example.

力争上游就是通过树立榜样来推动其他玩家做正确的事情。

It’s not about being the good guy, it’s about setting things up so that all of us can be the good guy.

这不是要成为好人，而是要做好准备，让我们所有人都能成为好人。

21:24 I’ll give a few examples of this.

我将举几个例子。

Early in the history of Anthropic, one of our co-founders, Chris Olah, who I believe you’re interviewing soon, he’s the co-founder of the field of mechanistic interpretability, which is an attempt to understand what’s going on inside AI models.

在 Anthropic 的早期历史中，我们的联合创始人之一 Chris Olah（我相信您很快就会采访他）是机械可解释性领域的联合创始人，该领域试图了解人工智能模型内部发生的事情。

So we had him and one of our early teams focus on this area of interpretability, which we think is good for making models safe and transparent.

因此，我们让他和我们的一个早期团队专注于可解释性这一领域，我们认为这有利于使模型安全和透明。

21:48 For three or four years that had no commercial application whatsoever.

三四年来，它没有任何商业应用。

It still doesn’t.

但还是不行。

Today we’re doing some early betas with it, and probably it will eventually, but this is a very, very long research bed, and one in which we’ve built in public and shared our results publicly.

今天我们正在用它做一些早期的测试，也许最终会的，但这是一个非常非常长的研究床，我们在其中公开构建并公开分享我们的结果。

And we did this because we think it’s a way to make models safer.

我们这样做是因为我们认为这是使模型更安全的一种方法。

An interesting thing is that as we’ve done this, other companies have started doing it as well.

有趣的是，当我们这样做时，其他公司也开始这样做。

In some cases because they’ve been inspired by it, in some cases because they’re worried that if other companies are doing this, look more responsible, they want to look more responsible too.

在某些情况下，因为他们受到了它的启发，在某些情况下，因为他们担心，如果其他公司这样做，看起来更负责任，他们也希望看起来更负责任。

No one wants to look like the irresponsible actor.

没有人希望自己看起来像一个不负责任的演员。

And so they adopt this as well.

所以他们也采用了这一点。

When folks come to Anthropic, interpretability is often a draw, and I tell them, “The other places you didn’t go, tell them why you came here.” And then you see soon that there’s interpretability teams elsewhere as well.

当人们来到 Anthropic 时，可解释性往往是一个吸引力，我告诉他们，“你没有去过的其他地方，告诉他们你为什么来到这里。”然后你很快就会发现其他地方也有可解释性团队。

22:47 And in a way that takes away our competitive advantage, because it’s like, “Oh, now others are doing it as well.” But it’s good for the broader system, and so we have to invent some new thing that we’re doing that others aren’t doing as well.

在某种程度上，这剥夺了我们的竞争优势，因为这就像，“哦，现在其他人也在这样做。”但这对更广泛的系统有好处，因此我们必须发明一些我们正在做而其他人没有做的新事情。

And the hope is to basically bid up the importance of doing the right thing.

我们希望从根本上提高做正确事的重要性。

And it’s not about us in particular.

这与我们无关。

It’s not about having one particular good guy.

这并不是关于拥有一个特别好的人。

Other companies can do this as well.

其他公司也可以这样做。

If they join the race to do this, that’s the best news ever.

如果他们加入这场竞赛，那就是有史以来最好的消息。

It’s about shaping the incentives to point upward instead of shaping the incentives to point downward.

这是关于塑造向上的激励，而不是塑造向下的激励。

And we should say this example of the field of mechanistic interpretability is just a rigorous non-hand wavy wave doing AI safety-

而我们应该说这个机械可解释性领域的例子只是一个严格的非手波做AI安全的波——

Yes.

是的。

…

……

23:34 or it’s tending that way.

或者它正趋向于这样。

Trying to.

正在努力。

23:36 I mean, I think we’re still early in terms of our ability to see things, but I’ve been surprised at how much we’ve been able to look inside these systems and understand what we see.

我的意思是，我认为我们在观察事物的能力方面还处于早期阶段，但我对我们能够深入了解这些系统并理解我们所看到的东西感到惊讶。

Unlike with the scaling laws where it feels like there’s some law that’s driving these models to perform better, on the inside, the models aren’t…

与缩放法则不同，在缩放法则中，感觉有某种法则在推动这些模型表现得更好，但从内部来看，这些模型并不是……

There’s no reason why they should be designed for us to understand them, right?

它们没有理由被设计成让我们理解它们，对吗？

They’re designed to operate, they’re designed to work.

它们是为操作而设计的，为工作而设计。

Just like the human brain or human biochemistry.

就像人脑或人体生物化学一样。

They’re not designed for a human to open up the hatch, look inside and understand them.

它们并不是为人类打开舱门、观察内部并理解它们而设计的。

But we have found, and you can talk in much more detail about this to Chris, that when we open them up, when we do look inside them, we find things that are surprisingly interesting.

但我们发现，你可以与克里斯更详细地谈论这一点，当我们打开它们，当我们确实观察它们的内部时，我们会发现一些令人惊讶的有趣的东西。

And as a side effect, you also get to see the beauty of these models.

作为副作用，您还可以看到这些模型的美丽。

24:20 You get to explore the beautiful nature of large neural networks through the MEC and TERP kind of methodology.

您可以通过 MEC 和 TERP 类型的方法来探索大型神经网络的美丽本质。

I’m amazed at how clean it’s been.

我对它的干净程度感到惊讶。

24:29 I’m amazed at things like induction heads.

我对感应头之类的东西感到惊讶。

I’m amazed at things like that we can use sparse auto-encoders to find these directions within the networks, and that the directions correspond to these very clear concepts.

我很惊讶我们可以使用稀疏自动编码器在网络中找到这些方向，并且这些方向对应于这些非常清晰的概念。

24:49 We demonstrated this a bit with the Golden Gate Bridge Claude.

我们用金门大桥克劳德演示了这一点。

So this was an experiment where we found a direction inside one of the neural networks layers that corresponded to the Golden Gate Bridge.

所以这是一个实验，我们在神经网络层之一内找到了与金门大桥相对应的方向。

And we just turned that way up.

我们就这样转向了。

And so we released this model as a demo, it was kind of half a joke, for a couple days, but it was illustrative of the method we developed.

所以我们发布了这个模型作为演示，几天来这有点半开玩笑，但它说明了我们开发的方法。

And you could take the model, you could ask it about anything.

你可以使用这个模型，你可以向它询问任何事情。

It would be like you could say, “How was your day?” And anything you asked, because this feature was activated, it would connect to the Golden Gate Bridge.

就像你可以说：“今天过得怎么样？”无论你问什么，因为这个功能被激活，它都会连接到金门大桥。

So it would say, I’m feeling relaxed and expansive, much like the arches of the Golden Gate Bridge, or-

所以它会说，我感到放松和广阔，就像金门大桥的拱门，或者-

It would masterfully change topic to the Golden Gate Bridge and integrate it.

它将巧妙地将话题转移到金门大桥并将其整合起来。

25:31 There was also a sadness to the focus it had on the Golden Gate Bridge.

它对金门大桥的关注也令人悲伤。

I think people quickly fell in love with it, I think.

我认为人们很快就爱上了它。

So people already miss it, because it was taken down, I think after a day.

所以人们已经想念它了，因为我想一天后它就被删除了。

Somehow these interventions on the model, where you kind of adjust its behavior, somehow emotionally made it seem more human than any other version of the model.

不知何故，对模型的这些干预，即你调整其行为的方式，在情感上使其看起来比任何其他版本的模型都更加人性化。

It’s a strong personality, strong identity.

这是一种强烈的个性，强烈的认同。

It has a strong personality.

它有很强的个性。

25:58 It has these kind of obsessive interests.

它有这种强迫性的兴趣。

We can all think of someone who’s obsessed with something.

我们都可以想到一个痴迷于某件事的人。

So it does make it feel somehow a bit more human.

所以它确实让人感觉更加人性化。

Let’s talk about the present.

我们来谈谈现在吧。

26:08 Let’s talk about Claude.

我们来谈谈克劳德。

So this year, a lot has happened.

所以今年发生了很多事情。

In March.

三月份。

Claude 3 Opus, Sonnet, Haiku were released.

克劳德第三部作品、十四行诗、俳句发行。

Then Claude 3.5 Sonnet in July, with an updated version just now released.

然后是 7 月份的 Claude 3.5 Sonnet，刚刚发布了更新版本。

And then also Claude 3.5 Haiku was released.

随后还发布了 Claude 3.5 Haiku。

Okay.

好的。

Can you explain the difference between Opus, Sonnet and Haiku, and how we should think about the different versions?

您能否解释一下 Opus、Sonnet 和 Haiku 之间的区别，以及我们应该如何看待不同版本？

Yeah, so let’s go back to March when we first released these three models.

是的，让我们回到三月份我们首次发布这三款车型的时候。

26:34 So our thinking was different companies produce large and small models, better and worse models.

所以我们的想法是不同的公司生产大和小的模型，更好和更差的模型。

We felt that there was demand, both for a really powerful model, and that might be a little bit slower that you’d have to pay more for, and also for fast cheap models that are as smart as they can be for how fast and cheap.

我们认为存在需求，既需要真正强大的模型，而且可能会慢一点，你必须支付更多费用，也需要快速便宜的模型，因为它们可以快速和便宜地智能。

Whenever you want to do some kind of difficult analysis, like if I want to write code for instance, or I want to brainstorm ideas or I want to do creative writing, I want the really powerful model.

每当你想做某种困难的分析时，例如，如果我想编写代码，或者我想集思广益，或者我想做创意写作，我想要真正强大的模型。

27:15 But then there’s a lot of practical applications in a business sense where it’s like I’m interacting with a website, I am doing my taxes, or I’m talking to a legal advisor and I want to analyze a contract.

但在商业意义上还有很多实际应用，比如我正在与网站交互，我正在报税，或者我正在与法律顾问交谈并且我想分析一份合同。

Or we have plenty of companies that are just like, I want to do auto-complete on my IDE or something.

或者我们有很多公司都说，我想在我的 IDE 上进行自动完成或其他功能。

And for all of those things, you want to act fast and you want to use the model very broadly.

对于所有这些事情，您想要快速采取行动，并且想要非常广泛地使用该模型。

So we wanted to serve that whole spectrum of needs.

所以我们想满足所有的需求。

So we ended up with this kind of poetry theme.

所以我们最终有了这样的诗歌主题。

And so what’s a really short poem?

那么什么是真正的短诗呢？

It’s a haiku.

这是一首俳句。

Haiku is the small, fast, cheap model that was at the time, was really surprisingly intelligent for how fast and cheap it was.

Haiku 是当时体积小、速度快、价格便宜的型号，它的速度和价格确实令人惊讶地智能。

28:03 Sonnet is a medium-sized poem, write a couple paragraphs.

十四行诗是一首中等大小的诗，写几个段落。

And so Sonnet was the middle model.

所以十四行诗是中间模型。

It is smarter but also a little bit slower, a little bit more expensive.

它更智能，但也有点慢，也更贵。

And Opus, like a Magnum Opus is a large work, Opus was the largest, smartest model at the time.

而Opus，就像Magnum Opus 是一部大型作品一样，Opus 是当时最大、最智能的型号。

So that was the original kind of thinking behind it.

这就是其背后最初的想法。

28:24 And our thinking then was, “Well, each new generation of models should shift that trade- off curve.” So when we released Sonnet 3.5, it has roughly the same cost and speed as the Sonnet 3 model, but it increased its intelligence to the point where it was smarter than the original Opus 3 model.

当时我们的想法是，“嗯，每一代新一代模型都应该改变这种权衡曲线。”因此，当我们发布 Sonnet 3.5 时，它的成本和速度与 Sonnet 3 型号大致相同，但它提高了智能性，比原来的 Opus 3 型号更智能。

Especially for code, but also just in general.

特别是对于代码而言，但也只是一般而言。

And so now we’ve shown results for Haiku 3.5.

现在我们已经展示了俳句 3.5 的结果。

And I believe Haiku 3.5, the smallest new model, is about as good as Opus 3, the largest old model.

我相信，最小的新型号 Haiku 3.5 与最大的旧型号 Opus 3 差不多。

So basically the aim here is to shift the curve and then at some point there’s going to be an Opus 3.5.

所以基本上我们的目标是改变曲线，然后在某个时候将会出现 Opus 3.5。

29:13 Now every new generation of models has its own thing.

现在每一代新一代车型都有自己的东西。

They use new data, their personality changes in ways that we try to steer but are not fully able to steer.

他们使用新数据，他们的性格发生变化，我们试图引导但无法完全引导。

And so there’s never quite that exact equivalence, where the only thing you’re changing is intelligence.

因此，永远不会有那么精确的对等，你唯一要改变的就是智力。

We always try and improve other things and some things change without us knowing or measuring.

我们总是尝试改进其他事情，但有些事情在我们不知道或衡量的情况下发生了变化。

So it’s very much an inexact science.

所以这是一门非常不精确的科学。

In many ways, the manner and personality of these models is more an art than it is a science.

从很多方面来说，这些模特的风格和个性与其说是一门科学，不如说是一门艺术。

So what is the reason for the span of time between say, Claude Opus 3.0 and 3.5?

那么 Claude Opus 3.0 和 3.5 之间时间跨度的原因是什么？

29:44 What takes that time, if you can speak to it?

如果你能说话的话，那需要什么时间呢？

Yeah, so there’s different processes.

是的，所以有不同的流程。

29:58 There’s pre-training, which is just kind of the normal language model training.

有预训练，这只是正常的语言模型训练。

And that takes a very long time.

这需要很长时间。

That uses, these days, tens of thousands, sometimes many tens of thousands of GPUs or TPUs or training them, or we use different platforms, but accelerator chips, often training for months.

如今，需要使用数万个、有时甚至数万个 GPU 或 TPU 或训练它们，或者我们使用不同的平台，但使用加速器芯片，通常训练数月。

30:26 There’s then a kind of post-training phase where we do reinforcement learning from human feedback as well as other kinds of reinforcement learning.

然后是一种训练后阶段，我们根据人类反馈以及其他类型的强化学习进行强化学习。

That phase is getting larger and larger now, and often that’s less of an exact science.

现在这个阶段变得越来越大，而且通常不再是一门精确的科学。

It often takes effort to get it right.

通常需要付出努力才能做到正确。

Models are then tested with some of our early partners to see how good they are, and they’re then tested, both internally and externally, for their safety, particularly for catastrophic and autonomy risks.

然后与我们的一些早期合作伙伴一起测试模型，看看它们有多好，然后对它们进行内部和外部测试，以确保其安全性，特别是灾难性和自治风险。

So we do internal testing according to our responsible scaling policy, which I could talk more about that in detail.

因此，我们根据我们负责任的扩展政策进行内部测试，我可以详细讨论这一点。

31:06 And then we have an agreement with the US and the UK AI Safety Institute, as well as other third-party testers in specific domains, to test the models for what are called CBRN risks, chemical, biological, radiological, and nuclear.

然后我们与美国和英国人工智能安全研究所以及特定领域的其他第三方测试机构达成协议，测试模型的 CBRN 风险、化学、生物、放射性和核风险。

We don’t think that models pose these risks seriously yet, but every new model we want to evaluate to see if we’re starting to get close to some of these more dangerous capabilities.

我们认为模型还没有严重构成这些风险，但我们想要评估每个新模型，看看我们是否开始接近其中一些更危险的功能。

So those are the phases, and then it just takes some time to get the model working in terms of inference and launching it in the API.

这些就是这些阶段，然后只需要一些时间即可让模型在推理方面发挥作用并在 API 中启动它。

So there’s just a lot of steps to actually making a model work.

因此，要使模型真正发挥作用，需要执行很多步骤。

And of course, we’re always trying to make the processes as streamlined as possible.

当然，我们一直在努力使流程尽可能简化。

31:55 We want our safety testing to be rigorous, but we want it to be rigorous and to be automatic, to happen as fast as it can, without compromising on rigor.

我们希望我们的安全测试是严格的，但我们希望它是严格的、自动的、尽可能快地进行，而不影响严格性。

Same with our pre-training process and our post-training process.

与我们的训练前流程和训练后流程相同。

So it’s just building anything else.

所以它只是建造其他东西。

It’s just like building airplanes.

这就像建造飞机一样。

You want to make them safe, but you want to make the process streamlined.

您希望确保它们的安全，但您希望简化流程。

And I think the creative tension between those is an important thing in making the models work.

我认为这些之间的创造性张力对于使模型发挥作用非常重要。

Yeah, rumor on the street, I forget who was saying that, Anthropic has really good tooling.

是的，街上有传言，我忘了是谁说的，Anthropic 有非常好的工具。

32:20 So probably a lot of the challenge here is, on the software engineering side, is to build the tooling to have a efficient, low-friction interaction with the infrastructure.

因此，在软件工程方面，这里的很多挑战可能是构建工具以与基础设施进行高效、低摩擦的交互。

You would be surprised how much of the challenges of building these models comes down to software engineering, performance engineering.

您可能会惊讶地发现，构建这些模型的挑战有多少归结于软件工程和性能工程。

32:36 From the outside, you might think, “Oh man, we had this Eureka breakthrough.” You know, this movie with the science.

从外面看，你可能会想，“天哪，我们有了尤里卡突破。”你知道，这部电影充满了科学。

“We discovered it, we figured it out.” But I think all things, even incredible discoveries, they almost always come down to the details.

“我们发现了它，我们弄清楚了。”但我认为所有的事情，即使是令人难以置信的发现，它们几乎总是归结为细节。

And often super, super boring details.

而且往往是超级非常无聊的细节。

I can’t speak to whether we have better tooling than other companies.

我无法谈论我们是否拥有比其他公司更好的工具。

I mean, haven’t been at those other companies, at least not recently, but it’s certainly something we give a lot of attention to.

我的意思是，我没有去过其他公司，至少最近没有，但这确实是我们非常关注的事情。

I don’t know if you can say, but from Claude 3 to Claude 3.5, is there any extra pre-training going on, or is it mostly focused on the post-training?

不知道你能否说一下，从Claude 3到Claude 3.5，是否有额外的预训练，或者主要集中在后训练？

33:18 There’s been leaps in performance.

性能上有了飞跃。

Yeah, I think at any given stage, we’re focused on improving everything at once.

是的，我认为在任何特定阶段，我们都会专注于立即改进一切。

33:29 Just naturally.

很自然。

Like, there are different teams.

就像，有不同的团队。

Each team makes progress in a particular area, in making their particular segment of the relay race better.

每个团队都在特定领域取得进步，使接力赛的特定部分变得更好。

And it’s just natural that when we make a new model, we put all of these things in at once.

当我们制作一个新模型时，我们很自然地会立即将所有这些东西放入其中。

So the data you have, the preference data you get from RLHF, is there ways to apply it to newer models as it get trained up?

那么，您拥有的数据，即您从 RLHF 获得的偏好数据，是否有办法在训练后将其应用到新模型中？

Yeah.

是的。

34:00 Preference data from old models sometimes gets used for new models, although of course it performs somewhat better when it’s trained on the new models.

旧模型的偏好数据有时会用于新模型，尽管在新模型上训练时它的性能当然会更好。

Note that we have this constitutional AI method such that we don’t only use preference data, there’s also a post-training process where we train the model against itself.

请注意，我们有这种宪法性的人工智能方法，因此我们不仅使用偏好数据，还有一个训练后过程，我们可以针对自身来训练模型。

And there’s new types of post-training the model against itself that are used every day.

每天都会使用针对模型自身进行的新型后训练。

So it’s not just RLHF, a bunch of other methods as well.

所以不仅仅是 RLHF，还有很多其他方法。

Post-training, I think, is becoming more and more sophisticated.

我认为，后期培训正变得越来越复杂。

Well, what explains the big leap in performance for the new Sonnet 3.5, I mean, at least in the programming side?

好吧，我的意思是，至少在编程方面，如何解释新 Sonnet 3.5 性能的巨大飞跃？

34:30 And maybe this is a good place to talk about benchmarks.

也许这是讨论基准的好地方。

What does it mean to get better?

变得更好意味着什么？

Just the number went up, but I program, but I also love programming, and I Claude 3.5 through Cursor is what I use to assist me in programming.

只是数字上升了，但是我编程，而且我也热爱编程，我通过Cursor 3.5就是我用来辅助我编程的。

And there was, at least experientially, anecdotally, it’s gotten smarter at programming.

至少从经验上来看，它在编程方面变得更加聪明。

So what does it take to get it smarter?

那么怎样才能让它变得更聪明呢？

We-

我们——

So what does it take to get it smarter?

那么怎样才能让它变得更聪明呢？

We observe that as well.

我们也观察到这一点。

35:03 By the way, there were a couple very strong engineers here at Anthropic, who all previous code models, both produced by us and produced by all the other companies, hadn’t really been useful to them.

顺便说一句，Anthropic 有几个非常强大的工程师，他们以前的所有代码模型，无论是我们生产的还是所有其他公司生产的，对他们来说都没有真正的用处。

They said, “Maybe this is useful to a beginner.

他们说：“也许这对初学者有用。

It’s not useful to me.” But Sonnet 3.5, the original one for the first time, they said, “Oh, my God, this helped me with something that it would’ve taken me hours to do.

这对我来说没有用。”但是十四行诗 3.5，也就是第一次的原始版本，他们说：“哦，天哪，这帮助我完成了一些原本需要几个小时才能完成的事情。

This is the first model that’s actually saved me time.”

这是第一个真正节省了我时间的模型。”

35:31 So again, the water line is rising.

水位再次上升。

And then I think the new Sonnet has been even better.

然后我认为新的十四行诗甚至更好。

In terms of what it takes, I’ll just say it’s been across the board.

就需要什么而言，我只能说这是全面的。

It’s in the pre-training, it’s in the post-training, it’s in various evaluations that we do.

在预训练中，在训练后，在我们所做的各种评估中。

We’ve observed this as well.

我们也观察到了这一点。

And if we go into the details of the benchmark, so SWE-bench is basically…

如果我们深入了解基准测试的细节，那么 SWE-bench 基本上是……

Since you’re a programmer, you’ll be familiar with pull requests, and just pull requests, they’re like a sort of atomic unit of work.

由于您是一名程序员，您会熟悉拉取请求，并且只是拉取请求，它们就像一种原子工作单元。

You could say I’m implementing one thing.

你可以说我正在实现一件事。

36:12 So SWE-bench actually gives you a real world situation where the code base is in a current state and I’m trying to implement something that’s described in language.

因此，SWE-bench 实际上为您提供了一个现实世界的情况，其中代码库处于当前状态，并且我正在尝试实现用语言描述的东西。

We have internal benchmarks where we measure the same thing and you say, “Just give the model free rein to do anything, run anything, edit anything.

我们有内部基准，我们测量相同的东西，然后你说：“只要让模型自由地做任何事情，运行任何东西，编辑任何东西。

How well is it able to complete these tasks?” And it’s that benchmark that’s gone from “it can do it 3% of the time” to “it can do it about 50% of the time.”

完成这些任务的能力如何？”正是这个基准从“它可以在 3% 的时间完成”变成“它可以在大约 50% 的时间完成”。

36:43 So I actually do believe that you can gain benchmarks, but I think if we get to 100% on that benchmark in a way that isn’t over-trained or game for that particular benchmark, probably represents a real and serious increase in programming ability.

所以我实际上确实相信你可以获得基准，但我认为如果我们以一种没有过度训练或针对特定基准的方式达到 100%，可能代表着编程能力的真正而严重的提高。

And I would suspect that if we can get to 90, 95% that it will represent ability to autonomously do a significant fraction of software engineering tasks.

我怀疑，如果我们能够达到 90%、95%，这将代表我们有能力自主完成大部分软件工程任务。

Well, ridiculous timeline question.

好吧，可笑的时间线问题。

37:13 When is Claude Opus 3.5 coming up?

Claude Opus 3.5 何时推出？

Not giving you an exact date, but as far as we know, the plan is still to have a Claude 3.5 Opus.

没有给你确切的日期，但据我们所知，计划仍然是推出 Claude 3.5 Opus。

Are we going to get it before GTA 6 or no?

我们是否会在 GTA 6 之前获得它？

Like Duke Nukem Forever?

喜欢《永远的毁灭公爵》吗？

Duke Nukem.

毁灭公爵。

37:30 Right.

正确的。

What was that game?

那场比赛是什么？

37:32 There was some game that was delayed 15 years.

有一场比赛被推迟了15年。

That’s right.

这是正确的。

Was that Duke Nukem Forever?

这是永远的毁灭公爵吗？

Yeah.

是的。

37:36 And I think GTA is now just releasing trailers.

我认为 GTA 现在刚刚发布预告片。

It’s only been three months since we released the first Sonnet.

距离我们发布第一首十四行诗才过去了三个月。

Yeah, it’s the incredible pace of release.

是的，这是令人难以置信的发布速度。

It just tells you about the pace, the expectations for when things are going to come out.

它只是告诉你事情发生的速度和期望。

So what about 4.0?

那么4.0呢？

37:49 So how do you think, as these models get bigger and bigger, about versioning and also just versioning in general, why Sonnet 3.5 updated with the date?

那么，随着这些模型变得越来越大，关于版本控制以及一般版本控制，您如何看待为什么 Sonnet 3.5 会更新日期？

Why not Sonnet 3.6, which a lot of people are calling it?

为什么不是 Sonnet 3.6，很多人都这么称呼它呢？

Naming is actually an interesting challenge here, right?

命名实际上是一个有趣的挑战，对吧？

38:06 Because I think a year ago, most of the model was pre-training.

因为我想一年前，大部分模型都是预训练的。

And so you could start from the beginning and just say, “Okay, we’re going to have models of different sizes.

所以你可以从头开始，然后说：“好吧，我们将有不同尺寸的模型。

We’re going to train them all together and we’ll have a family of naming schemes and then we’ll put some new magic into them and then we’ll have the next generation.”

我们将一起训练他们，我们将拥有一系列命名方案，然后我们将为他们注入一些新的魔力，然后我们将拥有下一代。”

38:26 The trouble starts already when some of them take a lot longer than others to train.

当他们中的一些人比其他人花费更长的时间来训练时，麻烦就开始了。

That already messes up your time a little bit.

这已经让你的时间有点混乱了。

But as you make big improvement in pre-training, then you suddenly notice, “Oh, I can make better pre-train model.” And that doesn’t take very long to do, but clearly it has the same size and shape of previous models.

但当你在预训练方面取得巨大进步时，你会突然注意到，“哦，我可以制作更好的预训练模型。”这并不需要很长时间，但显然它的尺寸和形状与以前的型号相同。

So I think those two together as well as the timing issues.

所以我认为这两者一起以及时间安排问题。

Any kind of scheme you come up with, the reality tends to frustrate that scheme, right?

无论你想出什么样的计划，现实往往会挫败该计划，对吗？

It tends to break out of the scheme.

它往往会打破这个计划。

39:04 It’s not like software where you can say, “Oh, this is 3.7, this is 3.8.” No, you have models with different trade-offs.

它不像软件，你可以说，“哦，这是 3.7，这是 3.8。”不，你有不同权衡的模型。

You can change some things in your models, you can change other things.

您可以更改模型中的某些内容，也可以更改其他内容。

Some are faster and slower at inference.

有些推理速度更快或更慢。

Some have to be more expensive, some have to be less expensive.

有些必须更贵，有些必须更便宜。

And so I think all the companies have struggled with this.

所以我认为所有公司都在努力解决这个问题。

I think we were in a good position in terms of naming when we had Haiku, Sonnet and Opus.

我认为当我们有俳句、十四行诗和作品时，我们在命名方面处于有利位置。

It was great, great start.

这是伟大的，伟大的开始。

We’re trying to maintain it, but it’s not perfect, so we’ll try and get back to the simplicity.

我们正在努力维护它，但它并不完美，所以我们会尝试回归简单性。

39:32 But just the nature of the field, I feel like no one’s figured out naming.

但就这个领域的性质而言，我觉得没有人能想出命名方法。

It’s somehow a different paradigm from normal software and so none of the companies have been perfect at it.

它在某种程度上与普通软件不同，因此没有一家公司在这方面做得完美。

It’s something we struggle with surprisingly much relative to how trivial it is for the grand science of training the models.

相对于训练模型这一宏大科学而言，这是多么微不足道的事情，这是我们令人惊讶的挣扎。

So from the user side, the user experience of the updated Sonnet 3.5 is just different than the previous June 2024 Sonnet 3.5.

所以从用户方面来说，更新后的Sonnet 3.5的用户体验与之前的2024年6月Sonnet 3.5只是有所不同。

40:03 It would be nice to come up with some kind of labeling that embodies that.

如果能想出某种体现这一点的标签那就太好了。

Because people talk about Sonnet 3.5, but now there’s a different one.

因为人们谈论 Sonnet 3.5，但现在有了不同的。

And so how do you refer to the previous one and the new one when there’s a distinct improvement?

那么，当有明显改进时，您如何参考前一个和新的呢？

It just makes conversation about it just challenging.

这只会让有关它的谈话变得充满挑战。

Yeah, yeah.

是啊是啊。

40:34 I definitely think this question of there are lots of properties of the models that are not reflected in the benchmarks.

我绝对认为这个问题是模型的许多属性没有反映在基准测试中。

I think that’s definitely the case and everyone agrees.

我认为情况确实如此，而且每个人都同意。

And not all of them are capabilities.

而且并非所有这些都是能力。

Models can be polite or brusque, they can be very reactive or they can ask you questions.

模特可以是礼貌的，也可以是粗暴的，他们可以是非常反应性的，或者他们可以问你问题。

They can have what feels like a warm personality or a cold personality.

他们的性格可能是温暖的，也可能是冷酷的。

They can be boring or they can be very distinctive like Golden Gate Claude was.

它们可能很无聊，也可能非常独特，就像金门克劳德那样。

41:10 And we have a whole team focused on, I think we call it Claude character.

我们整个团队都专注于，我想我们称之为克劳德性格。

Amanda leads that team and we’ll talk to you about that, but it’s still a very inexact science and often we find that models have properties that we’re not aware of.

阿曼达领导着这个团队，我们会和你讨论这一点，但这仍然是一门非常不精确的科学，我们经常发现模型具有我们不知道的属性。

The fact of the matter is that you can talk to a model 10,000 times and there are some behaviors you might not see just like with a human, right?

事实上，你可以与模型交谈 10,000 次，但有些行为你可能不会像人类一样看到，对吧？

41:36 I can know someone for a few months and not know that they have a certain skill or not know that there’s a certain side to them.

我可以认识一个人几个月，却不知道他们有某种技能，或者不知道他们有特定的一面。

And so I think we just have to get used to this idea.

所以我认为我们必须习惯这个想法。

And we’re always looking for better ways of testing our models to demonstrate these capabilities and also to decide which are the personality properties we want models to have and which we don’t want to have.

我们一直在寻找更好的方法来测试我们的模型，以展示这些功能，并决定我们希望模型拥有哪些个性属性，以及我们不希望拥有哪些个性属性。

That itself, the normative question, is also super interesting.

这个规范问题本身也非常有趣。

I got to ask you a question from Reddit.

我要在 Reddit 上问你一个问题。

From Reddit?

来自 Reddit？

42:04 Oh, boy.

哦，男孩。

There’s just this fascinating, to me at least, it’s a psychological social phenomenon where people report that Claude has gotten dumber for them over time.

至少对我来说，这是一种令人着迷的心理社会现象，人们报告说随着时间的推移，克劳德对他们来说变得越来越笨。

42:07 And so the question is, does the user complaint about the dumbing down of Claude 3.5 Sonnet hold any water?

那么问题来了，用户对 Claude 3.5 Sonnet 的简化的抱怨是否站得住脚？

So are these anecdotal reports a kind of social phenomena or is there any cases where Claude would get dumber?

那么这些轶事报道是一种社会现象还是有没有克劳德会变笨的情况呢？

So this actually doesn’t apply.

所以这实际上并不适用。

42:33 This isn’t just about Claude.

这不仅仅是克劳德的事。

I believe I’ve seen these complaints for every foundation model produced by a major company.

我相信我已经在大公司生产的每个基础模型上看到过这些抱怨。

People said this about GPT-4, they said it about GPT-4 Turbo.

人们这样评价 GPT-4，他们这样评价 GPT-4 Turbo。

So a couple things.

有几件事。

One, the actual weights of the model, the actual brain of the model, that does not change unless we introduce a new model.

第一，模型的实际权重，模型的实际大脑，除非我们引入新模型，否则不会改变。

There are just a number of reasons why it would not make sense practically to be randomly substituting in new versions of the model.

有多种原因导致在模型的新版本中随机替换实际上没有意义。

43:09 It’s difficult from an inference perspective and it’s actually hard to control all the consequences of changing the weights of the model.

从推理的角度来看这是很困难的，而且实际上很难控制改变模型权重的所有后果。

Let’s say you wanted to fine-tune the model, I don’t know, to say “certainly” less, which an old version of Sonnet used to do.

假设你想对模型进行微调，我不知道，可以说“当然”更少，这是旧版本的 Sonnet 所做的。

You actually end up changing 100 things as well.

实际上你最终也改变了 100 件事。

So we have a whole process for it and we have a whole process for modifying the model.

所以我们有一个完整的流程，我们有一个修改模型的完整流程。

We do a bunch of testing on it.

我们对其进行了大量测试。

We do a bunch of user testing in early customers.

我们在早期客户中进行了大量的用户测试。

43:36 So we both have never changed the weights of the model without telling anyone.

所以我们从来没有在没有告诉任何人的情况下改变过模型的权重。

And certainly, in the current setup, it would not make sense to do that.

当然，在当前的设置下，这样做是没有意义的。

Now, there are a couple things that we do occasionally do.

现在，我们偶尔会做一些事情。

One is sometimes we run A/B tests, but those are typically very close to when a model is being released and for a very small fraction of time.

一是有时我们会运行 A/B 测试，但这些测试通常非常接近模型发布的时间，并且持续的时间很小。

44:01 So the day before the new Sonnet 3.5, I agree we should have had a better name.

因此，在新 Sonnet 3.5 发布的前一天，我同意我们应该有一个更好的名字。

It’s clunky to refer to it.

参考它很笨拙。

There were some comments from people that it’s gotten a lot better and that’s because a fraction we’re exposed to an A/B test for those one or two days.

有人评论说情况已经好很多了，这是因为我们有一两天接受了 A/B 测试。

The other is that occasionally the system prompt will change.

另外就是偶尔系统提示会发生变化。

The system prompt can have some effects, although it’s unlikely to dumb down models, it’s unlikely to make them dumber.

系统提示可以产生一些效果，尽管它不太可能使模型变得愚蠢，但也不可能让它们变得更愚蠢。

44:32 And we’ve seen that while these two things, which I’m listing to be very complete, happened quite infrequently, the complaints for us and for other model companies about the model change, the model isn’t good at this, the model got more censored, the model was dumbed down.

我们已经看到，虽然这两件事（我列出的非常完整）很少发生，但我们和其他模型公司对模型更改的抱怨是，模型不擅长这一点，模型受到更多审查，模型被简化了。

Those complaints are constant and so I don’t want to say people are imagining it or anything, but the models are, for the most part, not changing.

这些抱怨是持续不断的，所以我不想说这是人们的想象或其他什么，但模型在很大程度上并没有改变。

If I were to offer a theory, I think it actually relates to one of the things I said before, which is that models are very complex and have many aspects to them.

如果我要提供一个理论，我认为它实际上与我之前所说的一件事有关，即模型非常复杂并且有很多方面。

And so often, if I ask the model a question, if I’m like, “Do task X” versus, “Can you do task X?” the model might respond in different ways.

通常，如果我问模型一个问题，我会说“做任务 X”而不是“你能做任务 X 吗？”模型可能会以不同的方式做出反应。

And so there are all kinds of subtle things that you can change about the way you interact with the model that can give you very different results.

因此，您可以改变与模型交互的方式，从而产生截然不同的结果。

45:33 To be clear, this itself is like a failing by us and by the other model providers that the models are just often sensitive to small changes in wording.

需要明确的是，这本身就像我们和其他模型提供商的一个失败，因为模型通常对措辞的微小变化很敏感。

It’s yet another way in which the science of how these models work is very poorly developed.

这是这些模型如何运作的科学发展非常不完善的另一种方式。

And so if I go to sleep one night and I was talking to the model in a certain way and I slightly changed the phrasing of how I talk to the model, I could get different results.

因此，如果我有一天晚上睡觉时以某种方式与模型交谈，并且稍微改变了与模型交谈的措辞，我可能会得到不同的结果。

45:58 So that’s one possible way.

所以这是一种可能的方式。

The other thing is, man, it’s just hard to quantify this stuff.

另一件事是，伙计，很难量化这些东西。

It’s hard to quantify this stuff.

很难量化这些东西。

I think people are very excited by new models when they come out and then as time goes on, they become very aware of their limitations.

我认为人们对新模型的出现感到非常兴奋，然后随着时间的推移，他们变得非常意识到它们的局限性。

So that may be another effect, but that’s all a very long-winded way of saying for the most part, with some fairly narrow exceptions, the models are not changing.

所以这可能是另一种影响，但这只是一种非常冗长的说法，在大多数情况下，除了一些相当有限的例外，模型没有改变。

I think there is a psychological effect.

我认为有心理作用。

46:22 You just start getting used to it, the baseline raises.

你刚刚开始习惯它，基线就会上升。

When people who have first gotten Wi-Fi on airplanes, it’s amazing, magic.

当人们第一次在飞机上使用 Wi-Fi 时，这真是太神奇了。

It’s amazing.

太棒了。

46:32 Yeah.

是的。

And then you start-

然后你开始——

And now I’m like, “I can’t get this thing to work.

现在我想，“我无法让这件事发挥作用。

46:33 This is such a piece of crap.”

这真是一坨屎。”

Exactly.

确切地。

46:36 So it’s easy to have the conspiracy theory of, “They’re making Wi-Fi slower and slower.” This is probably something I’ll talk to Amanda much more about, but another Reddit question, “When will Claude stop trying to be my pure tentacle grandmother imposing its moral worldview on me as a paying customer?

因此很容易产生“他们让 Wi-Fi 速度越来越慢”的阴谋论。这可能是我会和阿曼达更多讨论的事情，但另一个 Reddit 问题是，“克劳德什么时候才能停止试图成为我纯粹的触手祖母，将其道德世界观强加给我作为付费客户？

And also, what is the psychology behind making Claude overly apologetic?” So this reports about the experience, a different angle on the frustration.

还有，让克劳德过度道歉的心理到底是什么？”所以这篇报道是关于这次经历的，从一个不同的角度来描述挫败感。

It has to do with the character [inaudible 00:47:06].

这与角色有关 [听不清 00:47:06]。

Yeah, so a couple points on this first.

是的，首先要强调几点。

47:06 One is things that people say on Reddit and Twitter or X or whatever it is, there’s actually a huge distribution shift between the stuff that people complain loudly about on social media and what actually statistically users care about and that drives people to use the models.

一是人们在 Reddit、Twitter 或 X 等上所说的内容，人们在社交媒体上大声抱怨的内容与统计用户实际关心的内容之间实际上存在巨大的分布变化，这促使人们使用模型。

47:27 People are frustrated with things like the model not writing out all the code or the model just not being as good at code as it could be, even though it’s the best model in the world on code.

人们对诸如模型没有写出所有代码或模型在代码方面表现不佳之类的事情感到沮丧，尽管它是世界上最好的代码模型。

I think the majority of things are about that, but certainly a vocal minority raise these concerns, are frustrated by the model refusing things that it shouldn’t refuse or apologizing too much or just having these annoying verbal tics.

我认为大多数事情都是关于这一点的，但肯定有少数人提出了这些担忧，因为模型拒绝了不应该拒绝的事情，或者道歉太多，或者只是有这些恼人的言语抽搐，这让他们感到沮丧。

47:59 The second caveat, and I just want to say this super clearly because I think some people don’t know it, others know it, but forget it.

第二个警告，我只想说得非常清楚，因为我认为有些人不知道，其他人知道，但忘记了。

It is very difficult to control across the board how the models behave.

全面控制模型的行为非常困难。

You cannot just reach in there and say, “Oh, I want the model to apologize less.” You can do that.

你不能只是伸手去说：“哦，我希望模特少道歉。”你可以这么做。

You can include training data that says, “Oh, the model should apologize less.” But then in some other situation, they end up being super rude or overconfident in a way that’s misleading people.

您可以包含这样的训练数据：“哦，模型应该少道歉。”但在其他情况下，他们最终会变得超级粗鲁或过度自信，从而误导人们。

48:30 So there are all these trade-offs.

所以存在所有这些权衡。

For example, another thing is if there was a period during which models, ours and I think others as well, were too verbose, they would repeat themselves, they would say too much.

例如，另一件事是，如果有一个时期，我们的模型和我认为其他人的模型都过于冗长，他们会重复自己，他们会说太多。

You can cut down on the verbosity by penalizing the models for just talking for too long.

你可以通过惩罚模型说话时间太长来减少冗长的内容。

What happens when you do that, if you do it in a crude way, is when the models are coding, sometimes they’ll say, “Rest of the code goes here,” right?

当你这样做时，如果你以一种粗略的方式这样做，会发生什么，当模型编码时，有时他们会说，“其余的代码放在这里，”对吗？

48:58 Because they’ve learned that that’s the way to economize and that they see it.

因为他们知道这就是节约的方法，而且他们也看到了这一点。

And then so that leads the model to be so-called lazy in coding where they’re just like, “Ah, you can finish the rest of it.” It’s not because we want to save on compute or because the models are lazy during winter break or any of the other conspiracy theories that have come up.

然后，这导致模型在编码方面变得所谓的懒惰，他们就像是，“啊，你可以完成剩下的部分。”这并不是因为我们想节省计算量，也不是因为模型在寒假期间很懒，也不是因为出现了任何其他阴谋论。

Actually, it’s just very hard to control the behavior of the model, to steer the behavior of the model in all circumstances at once.

实际上，控制模型的行为、在所有情况下同时引导模型的行为是非常困难的。

49:28 There’s this whack- a-mole aspect where you push on one thing and these other things start to move as well that you may not even notice or measure.

这就像打地鼠一样，当你推动一件事时，其他事物也会开始移动，而你甚至可能没有注意到或测量到。

And so one of the reasons that I care so much about grand alignment of these AI systems in the future is actually, these systems are actually quite unpredictable.

因此，我如此关心这些人工智能系统未来的大调整的原因之一实际上是，这些系统实际上是相当不可预测的。

They’re actually quite hard to steer and control.

它们实际上很难驾驶和控制。

And this version we’re seeing today of you make one thing better, it makes another thing worse, I think that’s like a present day analog of future control problems in AI systems that we can start to study today.

我们今天看到的这个版本让一件事变得更好，它让另一件事变得更糟，我认为这就像人工智能系统中未来控制问题的现代模拟，我们今天就可以开始研究。

50:12 I think that difficulty in steering the behavior and making sure that if we push an AI system in one direction, it doesn’t push it in another direction in some other ways that we didn’t want.

我认为很难控制行为并确保如果我们将人工智能系统推向一个方向，它不会以我们不想要的其他方式将其推向另一个方向。

I think that’s an early sign of things to come, and if we can do a good job of solving this problem of you ask the model to make and distribute smallpox and it says no, but it’s willing to help you in your graduate level virology class, how do we get both of those things at once?

我认为这是未来事情的早期迹象，如果我们能够很好地解决这个问题，你要求模型制作和分发天花，它说不，但它愿意在你的研究生水平病毒学课程上帮助你，我们如何才能同时获得这两件事呢？

It’s hard.

这很难。

50:48 It’s very easy to go to one side or the other and it’s a multidimensional problem.

很容易偏向某一边或另一边，这是一个多维的问题。

And so I think these questions of shaping the model’s personality, I think they’re very hard.

所以我认为塑造模特个性的这些问题非常难。

I think we haven’t done perfectly on them.

我认为我们在这些方面还没有做得很完美。

I think we’ve actually done the best of all the AI companies, but still so far from perfect.

我认为我们实际上已经做到了所有人工智能公司中最好的，但仍远未达到完美。

51:08 And I think if we can get this right, if we can control the false positives and false negatives in this very controlled present day environment, we’ll be much better at doing it for the future when our worry is: will the models be super autonomous?

我认为，如果我们能够做到这一点，如果我们能够在当今这个非常受控的环境中控制误报和漏报，那么当我们担心的是：模型会超级自治时，我们会在未来做得更好吗？

Will they be able to make very dangerous things?

他们能够做出非常危险的事情吗？

Will they be able to autonomously build whole companies and are those companies aligned?

他们能够自主建立整个公司吗？这些公司是否一致？

So I think of this present task as both vexing but also good practice for the future.

因此，我认为当前的任务既令人烦恼，又是对未来的良好实践。

What’s the current best way of gathering user feedback?

目前收集用户反馈的最佳方式是什么？

51:40 Not anecdotal data, but just large-scale data about pain points or the opposite of pain points, positive things, so on?

不是轶事数据，而是关于痛点或痛点的反面、积极的事情等等的大规模数据？

Is it internal testing?

是内部测试吗？

Is it a specific group testing, A/B testing?

是特定的小组测试、A/B 测试吗？

What works?

什么有效？

So typically, we’ll have internal model bashings where all of Anthropic…

因此，通常情况下，我们会进行内部模型抨击，所有 Anthropic ......

51:59 Anthropic is almost 1,000 people.

Anthropic有近1000人。

People just try and break the model.

人们只是尝试打破这个模式。

They try and interact with it various ways.

他们尝试以各种方式与之互动。

We have a suite of evals for, “Oh, is the model refusing in ways that it couldn’t?” I think we even had a “certainly” eval because again, at one point, the model had this problem where it had this annoying tick where it would respond to a wide range of questions by saying, “Certainly, I can help you with that.

我们有一套评估，“哦，模型是否以它不能的方式拒绝？”我认为我们甚至进行了一次“当然”的评估，因为在某一时刻，该模型再次遇到了这个问题，它有一个恼人的勾选，它会通过说“当然，我可以帮助你”来回答各种各样的问题。

Certainly, I would be happy to do that.

当然，我很乐意这样做。

Certainly, this is correct.”

当然，这是正确的。”

52:34 And so we had a “certainly” eval, which is: how often does the model say certainly?

因此，我们进行了“肯定”评估，即：模型多久会说“肯定”？

But look, this is just a whack-a-mole.

但看，这只是打地鼠。

What if it switches from “certainly” to “definitely”?

如果从“肯定”变成“肯定”怎么办？

So every time we add a new eval and we’re always evaluating for all the old things, we have hundreds of these evaluations, but we find that there’s no substitute for a human interacting with it.

因此，每次我们添加新的评估并且总是对所有旧事物进行评估时，我们有数百个这样的评估，但我们发现没有什么可以替代人类与之交互。

52:56 And so it’s very much like the ordinary product development process.

所以这与普通的产品开发过程非常相似。

We have hundreds of people within Anthropic bash the model.

我们 Anthropic 内部有数百人抨击这个模型。

Then we do external A/B tests.

然后我们进行外部 A/B 测试。

Sometimes we’ll run tests with contractors.

有时我们会与承包商一起进行测试。

We pay contractors to interact with the model.

我们付钱给承包商来与模型互动。

So you put all of these things together and it’s still not perfect.

所以你把所有这些东西放在一起，它仍然不完美。

You still see behaviors that you don’t quite want to see.

您仍然会看到您不太想看到的行为。

You still see the model refusing things that it just doesn’t make sense to refuse.

你仍然会看到模型拒绝一些没有意义的事情。

53:25 But I think trying to solve this challenge, trying to stop the model from doing genuinely bad things that everyone agrees it shouldn’t do, everyone agrees that the model shouldn’t talk about, I don’t know, child abuse material.

但我认为试图解决这个挑战，试图阻止模型做每个人都认为不应该做的真正坏事，每个人都同意模型不应该谈论，我不知道，虐待儿童的材料。

Everyone agrees the model shouldn’t do that, but at the same time, that it doesn’t refuse in these dumb and stupid ways.

每个人都同意该模型不应该这样做，但同时，它不会以这些愚蠢的方式拒绝。

53:49 I think drawing that line as finely as possible, approaching perfectly, is still a challenge and we’re getting better at it every day, but there’s a lot to be solved.

我认为尽可能精细地画出这条线，接近完美，仍然是一个挑战，我们每天都在进步，但还有很多问题需要解决。

And again, I would point to that as an indicator of a challenge ahead in terms of steering much more powerful models.

我再次指出，这表明在引导更强大的模型方面面临着挑战。

Do you think Claude 4.0 is ever coming out?

你认为克劳德4.0会问世吗？

I don’t want to commit to any naming scheme because if I say here, “We’re going to have Claude 4 next year,” and then we decide that we should start over because there’s a new type of model, I don’t want to commit to it.

我不想承诺任何命名方案，因为如果我在这里说，“明年我们将推出 Claude 4”，然后我们决定应该重新开始，因为有一种新型模型，我不想承诺它。

54:11 I would expect in a normal course of business that Claude 4 would come after Claude 3.

我预计在正常情况下，克劳德 4 会出现在克劳德 3 之后。

5, but you never know in this wacky field.

5，但在这个古怪的领域你永远不知道会发生什么。

But this idea of scaling is continuing.

但这种扩展的想法仍在继续。

Scaling is continuing.

规模扩张仍在继续。

54:38 There will definitely be more powerful models coming from us than the models that exist today.

我们肯定会推出比现有模型更强大的模型。

That is certain.

这是肯定的。

Or if there aren’t, we’ve deeply failed as a company.

如果没有，我们作为一家公司就彻底失败了。

Okay.

好的。

54:49 Can you explain the responsible scaling policy and the AI safety level standards, ASL levels?

您能否解释一下负责任的扩展政策和 AI 安全级别标准、ASL 级别？

As much as I am excited about the benefits of these models, and we’ll talk about that if we talk about Machines of Loving Grace, I’m worried about the risks and I continue to be worried about the risks.

尽管我对这些模型的好处感到兴奋，并且如果我们谈论慈爱机器，我们也会谈论这一点，但我担心风险，而且我仍然担心风险。

54:55 No one should think that Machines of Loving Grace was me saying I’m no longer worried about the risks of these models.

没有人应该认为《爱的机器》是我在说我不再担心这些模型的风险。

I think they’re two sides of the same coin.

我认为它们是同一枚硬币的两面。

55:16 The power of the models and their ability to solve all these problems in biology, neuroscience, economic development, governance and peace, large parts of the economy, those come with risks as well, right?

模型的力量及其解决生物学、神经科学、经济发展、治理与和平、经济大部分领域的所有这些问题的能力，这些问题也伴随着风险，对吗？

With great power comes great responsibility.

权力越大，责任越大。

The two are paired.

两者是配对的。

Things that are powerful can do good things and they can do bad things.

强大的东西可以做好事，也可以做坏事。

I think of those risks as being in several different categories, perhaps the two biggest risks that I think about.

我认为这些风险分为几个不同的类别，也许是我认为的两个最大的风险。

And that’s not to say that there aren’t risks today that are important, but when I think of really the things that would happen on the grandest scale, one is what I call catastrophic misuse.

这并不是说今天不存在重要的风险，但当我想到真正会发生的最大规模的事情时，其中一个就是我所说的灾难性滥用。

55:59 These are misuse of the models in domains like cyber, bio, radiological, nuclear, things that could harm or even kill thousands, even millions of people if they really, really go wrong.

这些都是对网络、生物、放射性、核等领域模型的滥用，如果真的出了问题，可能会伤害甚至杀死数千人，甚至数百万人。

These are the number one priority to prevent.

这些是要预防的第一要务。

And here I would just make a simple observation, which is that the models, if I look today at people who have done really bad things in the world, I think actually humanity has been protected by the fact that the overlap between really smart, well-educated people and people who want to do really horrific things has generally been small.

在这里我只想做一个简单的观察，那就是，如果我今天看看那些在世界上做过真正坏事的人，我认为实际上人类已经受到这样一个事实的保护：真正聪明、受过良好教育的人和想做真正可怕的事情的人之间的重叠通常很小。

56:44 Let’s say I’m someone who I have a PhD in this field, I have a well-paying job.

假设我拥有该领域的博士学位，并且有一份高薪工作。

There’s so much to lose.

有太多东西可以失去。

Even assuming I’m completely evil, which most people are not, why would such a person risk their life, risk their legacy, their reputation to do something truly, truly evil?

即使假设我是完全邪恶的（大多数人不是），为什么这样的人会冒着生命危险，冒着他们的遗产和声誉的危险去做一些真正、真正邪恶的事情呢？

If we had a lot more people like that, the world would be a much more dangerous place.

如果我们有更多这样的人，世界将变得更加危险。

And so my worry is that by being a much more intelligent agent, AI could break that correlation.

所以我担心，通过成为一个更加智能的代理，人工智能可能会打破这种相关性。

57:21 And so I do have serious worries about that.

所以我确实对此非常担心。

I believe we can prevent those worries.

我相信我们可以避免这些担忧。

But I think as a counterpoint to Machines of Loving Grace, I want to say that there’s still serious risks.

但我认为，与《慈爱机器》相反，我想说的是，它仍然存在严重的风险。

And the second range of risks would be the autonomy risks, which is the idea that models might, on their own, particularly as we give them more agency than they’ve had in the past, particularly as we give them supervision over wider tasks like writing whole code bases or someday even effectively operating entire companies, they’re on a long enough leash.

第二类风险是自主风险，即模型本身可能会受到足够长的束缚，特别是当我们赋予它们比过去更多的代理权时，特别是当我们让它们监督更广泛的任务时，比如编写整个代码库，或者有一天甚至有效地运营整个公司，它们会受到足够长的束缚。

Are they doing what we really want them to do?

他们正在做我们真正希望他们做的事情吗？

58:00 It’s very difficult to even understand in detail what they’re doing, let alone control it.

甚至很难详细了解他们在做什么，更不用说控制它了。

And like I said, these early signs that it’s hard to perfectly draw the boundary between things the model should do and things the model shouldn’t do that if you go to one side, you get things that are annoying and useless and you go to the other side, you get other behaviors.

就像我说的，这些早期迹象表明很难完美地划定模型应该做的事情和模型不应该做的事情之间的界限，如果你走到一侧，你会得到恼人且无用的东西，而你走到另一边，你会得到其他行为。

If you fix one thing, it creates other problems.

如果你解决了一件事，就会产生其他问题。

58:25 We’re getting better and better at solving this.

我们在解决这个问题方面做得越来越好。

I don’t think this is an unsolvable problem.

我不认为这是一个无法解决的问题。

I think this is a science like the safety of airplanes or the safety of cars or the safety of drugs.

我认为这是一门科学，就像飞机的安全性、汽车的安全性或药物的安全性一样。

I don’t think there’s any big thing we’re missing.

我认为我们没有遗漏任何重要的东西。

I just think we need to get better at controlling these models.

我只是认为我们需要更好地控制这些模型。

And so these are the two risks I’m worried about.

这是我担心的两个风险。

And our responsible scaling plan, which I’ll recognize is a very long-winded answer to your question.

我们负责任的扩展计划，我承认这是对你的问题的一个非常冗长的答案。

I love it.

我喜欢它。

58:49 I love it.

我喜欢它。

Our responsible scaling plan is designed to address these two types of risks.

我们负责任的扩展计划旨在解决这两类风险。

58:51 And so every time we develop a new model, we basically test it for its ability to do both of these bad things.

因此，每次我们开发一个新模型时，我们基本上都会测试它做这两件事的能力。

So if I were to back up a little bit, I think we have an interesting dilemma with AI systems where they’re not yet powerful enough to present these catastrophes.

因此，如果我退一步说，我认为人工智能系统面临着一个有趣的困境，它们还没有强大到足以呈现这些灾难。

I don’t know if they’ll ever present these catastrophes.

我不知道他们是否会呈现这些灾难。

It’s possible they won’t.

他们可能不会。

59:22 But the case for worry, the case for risk is strong enough that we should act now and they’re getting better very, very fast.

但令人担忧和风险的理由足够充分，我们应该立即采取行动，而且情况正在变得非常非常快。

I testified in the Senate that we might have serious bio risks within two to three years.

我在参议院作证说，我们可能在两到三年内面临严重的生物风险。

That was about a year ago.

那是大约一年前的事了。

Things have proceeded apace.

事情进展很快。

So we have this thing where it’s surprisingly hard to address these risks because they’re not here today, they don’t exist.

因此，解决这些风险非常困难，因为它们今天不存在，不存在。

They’re like ghosts, but they’re coming at us so fast because the models are improving so fast.

它们就像幽灵一样，但它们来得如此之快，因为模型改进得如此之快。

59:56 So how do you deal with something that’s not here today, doesn’t exist, but is coming at us very fast?

那么，你如何处理今天不存在、不存在但很快就会向我们袭来的东西呢？

So the solution we came up with for that, in collaboration with people like the organization METR and Paul Christiano is what you need for that are you need tests to tell you when the risk is getting close.

因此，我们与 METR 和 Paul Christiano 等组织合作提出的解决方案正是您所需要的，因为您需要测试来告诉您风险何时接近。

You need an early warning system.

你需要一个预警系统。

And so every time we have a new model, we test it for its capability to do these CBRN tasks as well as testing it for how capable it is of doing tasks autonomously on its own.

因此，每次我们有一个新模型时，我们都会测试它执行这些 CBRN 任务的能力，并测试它自主执行任务的能力。

1:00:35 And in the latest version of our RSP, which we released in the last month or two, the way we test autonomy risks is the AI model’s ability to do aspects of AI research itself, which when the AI models can do AI research, they become truly, truly autonomous.

在我们最近一两个月发布的最新版本的 RSP 中，我们测试自主风险的方式是人工智能模型本身进行人工智能研究的能力，当人工智能模型可以进行人工智能研究时，它们就会变得真正、真正的自主。

And that threshold is important for a bunch of other ways.

这个门槛对于许多其他方式都很重要。

And so what do we then do with these tasks?

那么我们该如何处理这些任务呢？

The RSP basically develops what we’ve called an if-then structure, which is if the models pass a certain capability, then we impose a certain set of safety and security requirements on them.

RSP 基本上开发了我们所说的 if-then 结构，即如果模型通过了某种功能，那么我们就会对它们施加一组特定的安全要求。

1:01:16 So today’s models are what’s called ASL-2.

所以今天的模型就是所谓的 ASL-2。

Models that were ASL-1 is for systems that manifestly don’t pose any risk of autonomy or misuse.

ASL-1 模型适用于显然不会带来任何自主或滥用风险的系统。

So for example, a chess playing bot, Deep Blue would be ASL-1.

例如，下棋机器人 Deep Blue 就是 ASL-1。

It’s just manifestly the case that you can’t use Deep Blue for anything other than chess.

很明显，你不能将深蓝用于国际象棋以外的任何事情。

It was just designed for chess.

它只是为国际象棋而设计的。

No one’s going to use it to conduct a masterful cyber attack or to run wild and take over the world.

没有人会用它来进行熟练的网络攻击或疯狂地统治世界。

1:01:47 ASL-2 is today’s AI systems where we’ve measured them and we think these systems are simply not smart enough to autonomously self-replicate or conduct a bunch of tasks and also not smart enough to provide meaningful information about CBRN risks and how to build CBRN weapons above and beyond what can be known from looking at Google.

ASL-2 是当今的人工智能系统，我们已经对其进行了测量，我们认为这些系统根本不够智能，无法自主进行自我复制或执行大量任务，而且也不够智能，无法提供有关 CBRN 风险以及如何制造 CBRN 武器的有意义的信息，超出了通过谷歌所能了解的信息。

In fact, sometimes they do provide information above and beyond a search engine, but not in a way that can be stitched together, not in a way that end-to-end is dangerous enough.

事实上，有时它们确实提供了搜索引擎之外的信息，但不是以可以拼接在一起的方式，也不是以端到端足够危险的方式。

1:02:26 So ASL-3 is going to be the point at which the models are helpful enough to enhance the capabilities of non-state actors, right?

因此，ASL-3 模型将足以帮助增强非国家行为者的能力，对吗？

State actors can already do, unfortunately, to a high level of proficiency, a lot of these very dangerous and destructive things.

不幸的是，国家行为者已经可以熟练地完成许多非常危险和破坏性的事情。

The difference is that non-state actors are not capable of it.

不同之处在于非国家行为者没有能力做到这一点。

And so when we get to ASL-3, we’ll take special security precautions designed to be sufficient to prevent theft of the model by non-state actors and misuse of the model as it’s deployed.

因此，当我们达到 ASL-3 时，我们将采取特殊的安全预防措施，足以防止非国家行为者盗窃模型以及模型部署时的滥用。

We’ll have to have enhanced filters targeted at these particular areas.

We’ll have to have enhanced filters targeted at these particular areas.

Cyber, bio, nuclear.

网络、生物、核。

Cyber, bio, nuclear and model autonomy, which is less a misuse risk and more a risk of the model doing bad things itself.

Cyber, bio, nuclear and model autonomy, which is less a misuse risk and more a risk of the model doing bad things itself.

1:03:09 ASL-4, getting to the point where these models could enhance the capability of a already knowledgeable state actor and/or become the main source of such a risk.

ASL-4，这些模型可以增强已经知识渊博的国家行为者的能力和/或成为此类风险的主要来源。

If you wanted to engage in such a risk, the main way you would do it is through a model.

如果你想承担这样的风险，主要的方法就是通过模型。

And then I think ASL-4 on the autonomy side, it’s some amount of acceleration in AI research capabilities with an AI model.

然后我认为 ASL-4 在自主方面，是通过人工智能模型在一定程度上加速人工智能研究能力。

1:03:45 And then ASL-5 is where we would get to the models that are truly capable that it could exceed humanity in their ability to do any of these tasks.

然后，ASL-5 是我们获得真正能够超越人类完成这些任务的模型的地方。

And so the point of the if-then structure commitment is basically to say, “Look, I don’t know, I’ve been working with these models for many years and I’ve been worried about risk for many years.

因此，“如果-那么”结构承诺的要点基本上是说，“听着，我不知道，我已经使用这些模型很多年了，多年来我一直担心风险。

It’s actually dangerous to cry wolf.

喊“狼来了”实际上是很危险的。

It’s actually dangerous to say this model is risky.

说这种模式有风险实际上是危险的。

And people look at it and they say this is manifestly not dangerous.” Again, it’s the delicacy of the risk isn’t here today, but it’s coming at us fast.

人们看着它，他们说这显然并不危险。”再说一次，危险的微妙之处虽然今天还没有出现，但它正在迅速向我们袭来。

1:04:27 How do you deal with that?

你如何处理这个问题？

It’s really vexing to a risk planner to deal with it.

对于风险规划者来说，处理这个问题确实很麻烦。

And so this if-then structure basically says, “Look, we don’t want to antagonize a bunch of people, we don’t want to harm our own ability to have a place in the conversation by imposing these very onerous burdens on models that are not dangerous today.” So the if-then, the trigger commitment is basically a way to deal with this.

因此，这个“如果-那么”结构基本上是说，“听着，我们不想对抗一群人，我们不想通过将这些非常繁重的负担强加给今天并不危险的模型来损害我们自己在对话中占有一席之地的能力。”因此，如果-那么，触发承诺基本上是解决这个问题的一种方法。

It says you clamp down hard when you can show the model is dangerous.

它说，当你可以证明模型是危险的时，你就必须严厉打击。

1:04:58 And of course, what has to come with that is enough of a buffer threshold that you’re not at high risk of missing the danger.

当然，随之而来的是足够的缓冲阈值，这样你就不会面临错过危险的高风险。

It’s not a perfect framework.

这不是一个完美的框架。

We’ve had to change it.

我们必须改变它。

We came out with a new one just a few weeks ago and probably going forward, we might release new ones multiple times a year because it’s hard to get these policies right technically, organizationally from a research perspective.

几周前我们推出了一项新政策，未来我们可能会每年多次发布新政策，因为从研究的角度来看，很难在技术上、组织上使这些政策正确。

But that is the proposal, if-then commitments and triggers in order to minimize burdens and false alarms now, but really react appropriately when the dangers are here.

但这就是建议、如果-那么承诺和触发因素，以便最大限度地减少现在的负担和误报，但当危险出现时真正做出适当的反应。

What do you think the timeline for ASL-3 is where several of the triggers are fired?

您认为 ASL-3 的几个触发器被触发的时间线是什么？

1:05:37 And what do you think the timeline is for ASL-4?

您认为 ASL-4 的时间表是什么？

Yeah.

是的。

1:05:44 So that is hotly debated within the company.

因此，公司内部对此展开了激烈的争论。

We are working actively to prepare ASL-3 security measures as well as ASL-3 deployment measures.

我们正在积极准备 ASL-3 安全措施以及 ASL-3 部署措施。

I’m not going to go into detail, but we’ve made a lot of progress on both and we’re prepared to be, I think, ready quite soon.

我不会详细说明，但我们在这两方面都取得了很大进展，我认为我们已经准备好很快做好准备。

I would not be surprised at all if we hit ASL-3 next year.

如果我们明年达到 ASL-3，我一点也不会感到惊讶。

There was some concern that we might even hit it this year.

有人担心我们今年可能会实现这一目标。

That’s still possible.

这还是有可能的。

That could still happen.

这种情况仍然可能发生。

It’s very hard to say, but I would be very, very surprised if it was 2030.

这很难说，但如果是 2030 年，我会非常非常惊讶。

I think it’s much sooner than that.

我认为比这早得多。

So there’s protocols for detecting it, the if-then and then there’s protocols for how to respond to it.

因此，有用于检测它的协议、“如果-那么”以及如何响应它的协议。

Yes.

是的。

How difficult is the second, the latter?

第二个，后者有多难？

Yeah.

是的。

1:06:34 I think for ASL-3, it’s primarily about security and about filters on the model relating to a very narrow set of areas when we deploy the model.

我认为对于 ASL-3，它主要涉及安全性以及模型上与我们部署模型时非常狭窄的一组区域相关的过滤器。

Because at ASL-3, the model isn’t autonomous yet.

因为在 ASL-3 中，该模型还不是自治的。

And so you don’t have to worry about the model itself behaving in a bad way even when it’s deployed internally.

因此，即使模型在内部部署，您也不必担心模型本身表现不佳。

So I think the ASL- 3 measures are, I won’t say straightforward, they’re rigorous, but they’re easier to reason about.

因此，我认为 ASL-3 措施很严格，但更容易推理。

1:07:06 I think once we get to ASL-4, we start to have worries about the models being smart enough that they might sandbag tests, they might not tell the truth about tests.

我认为一旦我们到达 ASL-4，我们就会开始担心模型是否足够聪明，以至于它们可能会进行沙袋测试，它们可能不会说出测试的真相。

We had some results came out about sleeper agents and there was a more recent paper about, “Can the models mislead attempts to sandbag their own abilities, present themselves as being less capable than they are?” And so I think with ASL-4, there’s going to be an important component of using other things than just interacting with the models.

我们得出了一些关于潜伏特工的结果，最近有一篇论文是关于“这些模型是否会误导试图掩盖自己能力的尝试，让自己表现得比实际能力差？”因此，我认为对于 ASL-4，除了与模型交互之外，还有使用其他东西的重要组成部分。

1:07:43 For example, interpretability or hidden chains of thought where you have to look inside the model and verify via some other mechanism that is not as easily corrupted as what the model says, that the model indeed has some property.

例如，可解释性或隐藏的思想链，您必须查看模型内部并通过其他一些不像模型所说的那么容易损坏的机制来验证模型确实具有某些属性。

So we’re still working on ASL-4.

所以我们仍在研究 ASL-4。

One of the properties of the RSP is that we don’t specify ASL-4 until we’ve hit ASL-3.

RSP 的特性之一是，在达到 ASL-3 之前我们不会指定 ASL-4。

And I think that’s proven to be a wise decision because even with ASL-3, again, it’s hard to know this stuff in detail, and we want to take as much time as we can possibly take to get these things right.

我认为这被证明是一个明智的决定，因为即使有了 ASL-3，也很难详细了解这些东西，我们希望花尽可能多的时间来把这些事情做好。

So for ASL-3, the bad actor will be the humans.

因此对于 ASL-3 来说，坏角色将是人类。

Humans, yes.

人类，是的。

And so there’s a little bit more…

还有更多……

For ASL- 4, it’s both, I think.

我认为对于 ASL-4 来说，两者兼而有之。

It’s both.

两者都是。

1:08:31 And so deception, and that’s where mechanistic interpretability comes into play, and hopefully the techniques used for that are not made accessible to the model.

因此，欺骗，这就是机械可解释性发挥作用的地方，希望模型无法使用用于欺骗的技术。

Yeah.

是的。

1:08:42 Of course, you can hook up the mechanistic interpretability to the model itself, but then you’ve lost it as a reliable indicator of the model state.

当然，您可以将机械可解释性与模型本身联系起来，但这样您就失去了它作为模型状态的可靠指标的作用。

There are a bunch of exotic ways you can think of that it might also not be reliable, like if the model gets smart enough that it can jump computers and read the code where you’re looking at its internal state.

你可以想到很多奇特的方法，但它也可能不可靠，比如如果模型足够智能，它可以跳过计算机并读取你正在查看其内部状态的代码。

We’ve thought about some of those.

我们已经考虑过其中的一些。

I think they’re exotic enough.

我认为它们足够具有异国情调。

There are ways to render them unlikely.

有一些方法可以让它们变得不可能。

But yeah, generally, you want to preserve mechanistic interpretability as a verification set or test set that’s separate from the training process of the model.

但是，一般来说，您希望保留机械可解释性作为验证集或测试集，与模型的训练过程分开。

See, I think as these models become better and better conversation and become smarter, social engineer becomes a threat too because they could start being very convincing to the engineers inside companies.

看，我认为随着这些模型变得越来越好对话并且变得越来越聪明，社会工程师也成为一种威胁，因为他们可以开始对公司内部的工程师非常有说服力。

Oh, yeah.

哦，是的。

1:09:30 Yeah.

是的。

We’ve seen lots of examples of demagoguery in our life from humans, and there’s a concern that models could do that as well.

我们在生活中看到过很多人类煽动的例子，人们担心模型也能做到这一点。

One of the ways that Claude has been getting more and more powerful is it’s now able to do some agentic stuff, computer use.

克劳德变得越来越强大的方式之一是它现在能够做一些代理的事情，即使用计算机。

1:09:40 There’s also an analysis within the sandbox of Claude.ai itself.

Claude.ai 本身的沙箱中也有一项分析。

But let’s talk about computer use.

但我们来谈谈计算机的使用。

That seems to me super exciting that you can just give Claude a task and it takes a bunch of actions, figures it out, and has access to the…

在我看来，这似乎非常令人兴奋，你可以给克劳德一个任务，它需要采取一系列行动，弄清楚它，并有权访问......

1:10:00 …

……

a bunch of actions, figures it out and has access to your computer through screenshots.

一系列操作，弄清楚并通过屏幕截图访问您的计算机。

So can you explain how that works and where that’s headed?

那么你能解释一下它是如何运作的以及它的发展方向吗？

Yeah.

是的。

1:10:10 It’s actually relatively simple.

其实比较简单。

So Claude has had for a long time, since Claude 3 back in March, the ability to analyze images and respond to them with text.

So Claude has had for a long time, since Claude 3 back in March, the ability to analyze images and respond to them with text.

The only new thing we added is those images can be screenshots of a computer and in response, we train the model to give a location on the screen where you can click and/or buttons on the keyboard, you can press in order to take action.

我们添加的唯一新内容是这些图像可以是计算机的屏幕截图，作为响应，我们训练模型在屏幕上给出一个位置，您可以在其中单击和/或键盘上的按钮，您可以按下以采取行动。

And it turns out that with actually not all that much additional training, the models can get quite good at that task.

事实证明，实际上不需要那么多额外的训练，模型就可以很好地完成这项任务。

It’s a good example of generalization.

这是概括的一个很好的例子。

People sometimes say if you get to lower earth orbit, you’re halfway to anywhere because of how much it takes to escape the gravity well.

人们有时会说，如果你到达近地轨道，那么你就已经到达了一半，因为要摆脱重力需要付出多少努力。

If you have a strong pre-trained model, I feel like you’re halfway to anywhere in terms of the intelligence space.

如果你有一个强大的预训练模型，我觉得你在智能领域已经成功了一半。

And so actually, it didn’t take all that much to get Claude to do this.

事实上，让克劳德这么做并没有花费太多的时间。

And you can just set that in a loop, give the model a screenshot, tell it what to click on, give it the next screenshot, tell it what to click on and that turns into a full kind of almost 3D video interaction of the model and it’s able to do all of these tasks.

你可以循环设置，给模型一个屏幕截图，告诉它要点击什么，给它下一个屏幕截图，告诉它要点击什么，这就会变成模型的一种完整的近 3D 视频交互，它能够完成所有这些任务。

We showed these demos where it’s able to fill out spreadsheets, it’s able to kind of interact with a website, it’s able to open all kinds of programs, different operating systems, Windows, Linux, Mac.

我们展示了这些演示，它能够填写电子表格，能够与网站进行交互，能够打开各种程序、不同的操作系统、Windows、Linux、Mac。

So I think all of that is very exciting.

所以我认为所有这些都非常令人兴奋。

I will say, while in theory there’s nothing you could do there that you couldn’t have done through just giving the model the API to drive the computer screen, this really lowers the barrier.

我想说的是，虽然从理论上讲，只要为模型提供 API 来驱动计算机屏幕，就没有什么是做不到的，但这确实降低了障碍。

And there’s a lot of folks who either aren’t in a position to interact with those APIs or it takes them a long time to do.

很多人要么无法与这些 API 进行交互，要么需要很长时间才能完成。

1:12:00 It’s just the screen is just a universal interface that’s a lot easier to interact with.

只是屏幕只是一个通用界面，更容易交互。

And so I expect over time, this is going to lower a bunch of barriers.

因此，我预计随着时间的推移，这将降低一系列障碍。

Now, honestly, the current model has, it leaves a lot still to be desired and we were honest about that in the blog.

现在，老实说，当前的模型仍然有很多不足之处，我们在博客中对此很诚实。

It makes mistakes, it misclicks.

它会犯错误，点击错误。

We were careful to warn people, “Hey, you can’t just leave this thing to run on your computer for minutes and minutes.

我们小心翼翼地警告人们，“嘿，你不能让这个东西在你的计算机上运行几分钟又几分钟。

You got to give this thing boundaries and guardrails.” And I think that’s one of the reasons we released it first in an API form rather than just hand the consumer and give it control of their computer.

You got to give this thing boundaries and guardrails.”我认为这就是我们首先以 API 形式发布它的原因之一，而不是仅仅交给消费者并让其控制他们的计算机。

But I definitely feel that it’s important to get these capabilities out there.

But I definitely feel that it’s important to get these capabilities out there.

As models get more powerful, we’re going to have to grapple with how do we use these capabilities safely.

随着模型变得越来越强大，我们将不得不努力解决如何安全地使用这些功能的问题。

How do we prevent them from being abused?

我们如何防止他们被虐待？

1:12:54 And I think releasing the model while the capabilities are still limited is very helpful in terms of doing that.

我认为在功能仍然有限的情况下发布模型对于做到这一点非常有帮助。

I think since it’s been released, a number of customers, I think Replit was maybe one of the most quickest to deploy things, have made use of it in various ways.

我认为自从它发布以来，许多客户（我认为 Replit 可能是部署速度最快的客户之一）已经以各种方式使用了它。

People have hooked up demos for Windows desktops, Macs, Linux machines.

People have hooked up demos for Windows desktops, Macs, Linux machines.

So yeah, it’s been very exciting.

是的，这非常令人兴奋。

I think as with anything else, it comes with new exciting abilities and then with those new exciting abilities, we have to think about how to make the model safe, reliable, do what humans want them to do.

我认为与其他任何事情一样，它具有新的令人兴奋的能力，然后有了这些新的令人兴奋的能力，我们必须考虑如何使模型安全、可靠，做人类希望他们做的事情。

It’s the same story for everything.

一切都是同一个故事。

Same thing.

同样的事情。

It’s that same tension.

也是同样的紧张。

But the possibility of use cases here, just the range is incredible.

但这里的用例的可能性，只是范围是令人难以置信的。

1:13:51 So how much to make it work really well in the future?

So how much to make it work really well in the future?

How much do you have to specially kind of go beyond what the pre-trained model is doing, do more post-training, RLHF or supervised fine-tuning or synthetic data just for the agentive stuff?

你需要在多大程度上超越预训练模型正在做的事情，做更多的训练后、RLHF 或监督微调或合成数据只是为了代理的东西？

Yeah.

是的。

1:14:10 I think speaking at a high level, it’s our intention to keep investing a lot in making the model better.

I think speaking at a high level, it’s our intention to keep investing a lot in making the model better.

I think we look at some of the benchmarks where previous models were like, “Oh, could do it 6% of the time,” and now our model would do it 14 or 22% of the time.

我认为我们会查看一些基准，以前的模型会说“哦，可以在 6% 的时间内完成”，而现在我们的模型可以在 14% 或 22% 的时间内完成。

And yeah, we want to get up to the human level reliability of 80, 90% just like anywhere else.

是的，我们希望像其他地方一样达到 80%、90% 的人类水平可靠性。

We’re on the same curve that we were on with SWE-bench where I think I would guess a year from now, the models can do this very, very reliably.

我们与 SWE-bench 处于同一曲线上，我想我猜一年后，模型可以非常非常可靠地做到这一点。

But you got to start somewhere.

但你必须从某个地方开始。

So you think it’s possible to get to the human level 90% basically doing the same thing you’re doing now or it has to be special for computer use?

所以你认为有可能达到 90% 的人类水平，基本上做你现在正在做的同样的事情，或者它必须是专门为计算机使用而设计的？

It depends what you mean by special and special in general, but I generally think the same kinds of techniques that we’ve been using to train the current model, I expect that doubling down on those techniques in the same way that we have for code, for models in general, for image input, for voice, I expect those same techniques will scale here as they have everywhere else,

这取决于你所说的特殊和一般特殊的含义，但我通常认为我们用来训练当前模型的技术相同，我希望以与代码、一般模型、图像输入、语音相同的方式加倍使用这些技术，我希望这些相同的技术在这里能够像在其他地方一样扩展，

But this is giving the power of action to Claude and so you could do a lot of really powerful things, but you could do a lot of damage also.

But this is giving the power of action to Claude and so you could do a lot of really powerful things, but you could do a lot of damage also.

Yeah, yeah.

是啊是啊。

1:15:27 No and we’ve been very aware of that.

不，我们非常清楚这一点。

Look, my view actually is computer use isn’t a fundamentally new capability like the CBRN or autonomy capabilities are.

Look, my view actually is computer use isn’t a fundamentally new capability like the CBRN or autonomy capabilities are.

It’s more like it kind of opens the aperture for the model to use and apply its existing abilities.

它更像是为模型打开了使用和应用其现有能力的窗口。

And so the way we think about it, going back to our RSP, is nothing that this model is doing inherently increases the risk from an RSP perspective, but as the models get more powerful, having this capability may make it scarier once it has the cognitive capability to do something at the ASL-3 and ASL-4 level, this may be the thing that kind of unbounds it from doing so.

因此，我们的思考方式是，回到 RSP，从 RSP 的角度来看，该模型本质上并没有增加风险，但随着模型变得越来越强大，一旦它具有在 ASL-3 和 ASL-4 级别做某事的认知能力，拥有这种能力可能会变得更可怕，这可能是一种让它无法这样做的事情。

So going forward, certainly this modality of interaction is something we have tested for and that we will continue to test for an RSP going forward.

因此，展望未来，这种交互方式肯定是我们已经测试过的，并且我们将继续测试 RSP 的未来。

I think it’s probably better to learn and explore this capability before the model is super capable

我认为在模型具有超能力之前学习和探索这种能力可能会更好

Yeah.

是的。

1:16:33 And there’s a lot of interesting attacks like prompt injection because now you’ve widened the aperture so you can prompt inject through stuff on screen.

还有很多有趣的攻击，比如提示注入，因为现在你已经扩大了孔径，这样你就可以通过屏幕上的东西提示注入。

So if this becomes more and more useful, then there’s more and more benefit to inject stuff into the model.

因此，如果这变得越来越有用，那么向模型中注入东西就会带来越来越多的好处。

If it goes to certain web page, it could be harmless stuff like advertisements or it could be harmful stuff, right?

If it goes to certain web page, it could be harmless stuff like advertisements or it could be harmful stuff, right?

Yeah, we’ve thought a lot about things like spam, CAPTCHA, mass…

Yeah, we’ve thought a lot about things like spam, CAPTCHA, mass…

1:16:53 One secret, I’ll tell you, if you’ve invented a new technology, not necessarily the biggest misuse, but the first misuse you’ll see, scams, just petty scams.

我告诉你一个秘密，如果你发明了一项新技术，不一定是最大的滥用，但你会看到的第一个滥用，骗局，只是小骗局。

Yeah.

是的。

It’s like a thing as old, people scamming each other, it’s this thing as old as time.

这就像一件古老的事情，人们互相欺骗，这件事和时间一样古老。

1:17:13 And it’s just every time, you got to deal with it.

每一次，你都必须处理它。

It’s almost silly to say, but it’s true, sort of bots and spam in general is a thing as it gets more and more intelligent-

It’s almost silly to say, but it’s true, sort of bots and spam in general is a thing as it gets more and more intelligent-

Yeah, yeah.

是啊，是啊。

…

……

1:17:29 it’s harder and harder to fight it.

对抗它变得越来越难。

Like I said, there are a lot of petty criminals in the world and it’s like every new technology is a new way for petty criminals to do something stupid and malicious.

就像我说的，世界上有很多小罪犯，似乎每一项新技术都是小罪犯做愚蠢和恶意事情的新方法。

Is there any ideas about sandboxing it?

有关于沙箱的任何想法吗？

1:17:45 How difficult is the sandboxing task?

沙盒任务有多困难？

Yeah, we sandbox during training.

是的，我们在训练期间使用沙箱。

1:17:49 So for example, during training we didn’t expose the model to the internet.

例如，在训练期间我们没有将模型暴露在互联网上。

I think that’s probably a bad idea during training because the model can be changing its policy, it can be changing what it’s doing and it’s having an effect in the real world.

我认为这在训练期间可能是一个坏主意，因为模型可以改变其策略，它可以改变它正在做的事情，并且它在现实世界中产生影响。

In terms of actually deploying the model, it kind of depends on the application.

In terms of actually deploying the model, it kind of depends on the application.

Sometimes you want the model to do something in the real world.

Sometimes you want the model to do something in the real world.

But of course, you can always put guard, you can always put guard rails on the outside.

但当然，你总是可以设置防护装置，你可以随时在外面设置护栏。

You can say, “Okay, well, this model’s not going to move data from my, the model’s not going to move any files from my computer or my web server to anywhere else.”

您可以说，“好吧，这个模型不会将数据从我的计算机或网络服务器上移动到其他任何地方。”

1:18:27 Now, when you talk about sandboxing, again, when we get to ASL-4, none of these precautions are going to make sense there.

现在，当你再次谈论沙箱时，当我们谈到 ASL-4 时，这些预防措施都没有意义。

When you talk about ASL-4, you’re then, the model is being, there’s theoretical worry the model could be smart enough to kind of break it to out of any box.

当你谈论 ASL-4 时，你会发现该模型存在理论上的担忧，该模型可能足够聪明，可以打破它的束缚。

And so there, we need to think about mechanistic interpretability.

因此，我们需要考虑机械的可解释性。

If we’re going to have a sandbox, it would need to be a mathematically provable.

如果我们想要一个沙箱，它就需要是一个可以数学证明的沙箱。

That’s a whole different world than what we’re dealing with with the models today.

这与我们今天处理的模型完全不同。

Yeah, the science of building a box from which ASL-4 AI system cannot escape.

Yeah, the science of building a box from which ASL-4 AI system cannot escape.

I think it’s probably not the right approach.

我认为这可能不是正确的方法。

1:19:08 I think the right approach, instead of having something unaligned that you’re trying to prevent it from escaping, I think it’s better to just design the model the right way or have a loop where you look inside the model and you’re able to verify properties and that gives you an opportunity to tell, iterate and actually get it right.

我认为正确的方法是，不要让某些未对齐的东西试图阻止它逃逸，我认为最好以正确的方式设计模型，或者有一个循环，让您查看模型内部，并且能够验证属性，这给您一个机会来告诉、迭代并真正做到正确。

I think containing bad models is a much worse solution than having good models.

我认为包含坏模型是比拥有好模型更糟糕的解决方案。

Let me ask about regulation.

我想问一下监管方面的问题。

1:19:36 What’s the role of regulation in keeping AI safe?

监管在确保人工智能安全方面发挥什么作用？

So for example, can you describe California AI regulation bill SB 1047 that was ultimately vetoed by the governor?

例如，您能描述一下最终被州长否决的加州人工智能监管法案 SB 1047 吗？

What are the pros and cons of this bill in general?

该法案总体上有哪些优点和缺点？

Yes, we ended up making some suggestions to the bill.

是的，我们最终对该法案提出了一些建议。

1:19:50 And then some of those were adopted and we felt, I think, quite positively about the bill by the end of that, it did still have some downsides.

然后其中一些被采纳，我认为，到最后我们对该法案相当积极，但它仍然有一些缺点。

And of course, it got vetoed.

当然，它被否决了。

I think at a high level, I think some of the key ideas behind the bill are I would say similar to ideas behind our RSPs.

我认为从较高的层面来看，该法案背后的一些关键想法与我们的 RSP 背后的想法类似。

And I think it’s very important that some jurisdiction, whether it’s California or the federal government and/or other countries and other states, passes some regulation like this.

我认为某些司法管辖区，无论是加利福尼亚州还是联邦政府和/或其他国家和其他州，通过一些这样的法规是非常重要的。

And I can talk through why I think that’s so important.

我可以谈谈为什么我认为这如此重要。

So I feel good about our RSP.

所以我对我们的 RSP 感觉很好。

It’s not perfect.

这并不完美。

It needs to be iterated on a lot.

需要进行多次迭代。

But it’s been a good forcing function for getting the company to take these risks seriously, to put them into product planning, to really make them a central part of work at Anthropic and to make sure that all of a thousand people, and it’s almost a thousand people now at Anthropic, understand that this is one of the highest priorities of the company, if not the highest priority.

但这是一个很好的强制功能，可以让公司认真对待这些风险，将其纳入产品规划，真正使它们成为 Anthropic 工作的核心部分，并确保所有 1000 名员工（现在 Anthropic 的近 1000 名员工）都明白，这是公司的最高优先事项之一，即使不是最高优先事项。

1:20:58 But one, there are still some companies that don’t have RSP like mechanisms, like OpenAI, Google did adopt these mechanisms a couple months after Anthropic did, but there are other companies out there that don’t have these mechanisms at all.

但其一，仍然有一些公司没有类似 RSP 的机制，比如 OpenAI，Google 在 Anthropic 几个月后就采用了这些机制，但也有其他公司根本没有这些机制。

And so if some companies adopt these mechanisms and others don’t, it’s really going to create a situation where some of these dangers have the property that it doesn’t matter if three out of five of the companies are being safe, if the other two are being unsafe, it creates this negative externality.

因此，如果一些公司采用这些机制，而另一些公司则不采用，这确实会造成一种情况，其中一些危险具有这样的性质：五分之三的公司是否安全并不重要，如果另外两家公司不安全，就会产生这种负外部性。

And I think the lack of uniformity is not fair to those of us who have put a lot of effort into being very thoughtful about these procedures.

我认为缺乏统一性对于我们这些付出了大量努力对这些程序进行深思熟虑的人来说是不公平的。

The second thing is I don’t think you can trust these companies to adhere to these voluntary plans on their own.

第二件事是，我认为你不能相信这些公司会自行遵守这些自愿计划。

Right?

正确的？

I like to think that Anthropic will, we do everything we can that we will, our RSP is checked by our long-term benefit trust, so we do everything we can to adhere to our own RSP.

我喜欢认为 Anthropic 会，我们会尽我们所能，我们的 RSP 由我们的长期利益信托来检查，所以我们会尽一切努力坚持我们自己的 RSP。

1:22:07 But you hear lots of things about various companies saying, “Oh, they said they would give this much compute and they didn’t.

但是你会听到很多关于不同公司的说法，“哦，他们说他们会提供这么多计算，但他们没有。

They said they would do this thing and the didn’t.” I don’t think it makes sense to litigate particular things that companies have done, but I think this broad principle that if there’s nothing watching over them, if there’s nothing watching over us as an industry, there’s no guarantee that we’ll do the right thing and the stakes are very high.

他们说他们会做这件事，但没有。”我认为对公司所做的特定事情提起诉讼是没有意义的，但我认为这个广泛的原则是，如果没有人监督他们，如果没有人监督我们这个行业，就不能保证我们会做正确的事，而且风险非常高。

And so I think it’s important to have a uniform standard that everyone follows and to make sure that simply that the industry does what a majority of the industry has already said is important and has already said that they definitely will do.

因此，我认为重要的是要有一个每个人都遵循的统一标准，并确保该行业只做该行业大多数人已经说过的重要的事情，并且已经说过他们肯定会做的事情。

1:22:52 Right, some people, I think there’s a class of people who are against regulation on principle.

对了，有些人，我觉得有一类人原则上是反对监管的。

I understand where that comes from.

我明白这是从哪里来的。

If you go to Europe and you see something like GDPR, you see some of the other stuff that they’ve done.

如果你去欧洲，你会看到 GDPR 之类的东西，你也会看到他们所做的其他一些事情。

Some of it’s good, but some of it is really unnecessarily burdensome and I think it’s fair to say really has slowed innovation.

其中一些是好的，但其中一些确实是不必要的负担，我认为公平地说确实减缓了创新。

And so I understand where people are coming from on priors.

所以我了解人们的先验来自哪里。

I understand why people start from that position.

我理解为什么人们从这个位置开始。

But again, I think AI is different.

但我再次认为人工智能是不同的。

If we go to the very serious risks of autonomy and misuse that I talked about just a few minutes ago, I think that those are unusual and they warrant an unusually strong response.

如果我们讨论我几分钟前谈到的自治和滥用的非常严重的风险，我认为这些都是不寻常的，因此需要做出异常强烈的反应。

And so I think it’s very important.

所以我认为这非常重要。

1:23:44 Again, we need something that everyone can get behind.

同样，我们需要每个人都可以支持的东西。

I think one of the issues with SB 1047, especially the original version of it was it had a bunch of the structure of RSPs, but it also had a bunch of stuff that was either clunky or that just would’ve created a bunch of burdens, a bunch of hassle and might even have missed the target in terms of addressing the risks.

我认为 SB 1047 的问题之一，尤其是它的原始版本，是它有一堆 RSP 的结构，但它也有一堆要么笨重，要么只会造成一堆负担、一堆麻烦，甚至可能没有达到解决风险的目标。

You don’t really hear about it on Twitter, you just hear about kind of people are cheering for any regulation.

你在推特上并没有真正听说过这件事，你只是听说有些人正在为任何监管欢呼。

And then the folks who are against make up these often quite intellectually dishonest arguments about how it’ll make us move away from California, bill doesn’t apply if you’re headquartered in California, bill only applies if you do business in California, or that it would damage the open source ecosystem or that it would cause all of these things.

然后，反对者会编造出这些在理智上相当不诚实的论点，说这将如何让我们离开加利福尼亚州，如果你的总部位于加利福尼亚州，该法案不适用，该法案仅适用于你在加利福尼亚州做生意的情况，或者它会损害开源生态系统，或者它会导致所有这些事情。

1:24:43 I think those were mostly nonsense, but there are better arguments against regulation.

我认为这些大多是无稽之谈，但还有更好的反对监管的论据。

There’s one guy, Dean Ball, who’s really, I think, a very scholarly analyst who looks at what happens when a regulation is put in place in ways that they can kind of get a life of their own or how they can be poorly designed.

我认为，迪恩·鲍尔（Dean Ball）确实是一位非常学术的分析师，他研究当监管到位时会发生什么，这些监管可以让他们获得自己的生活，或者他们如何设计得很糟糕。

And so our interest has always been we do think there should be regulation in this space, but we want to be an actor who makes sure that that regulation is something that’s surgical, that’s targeted at the serious risks and is something people can actually comply with.

因此，我们的兴趣一直是，我们确实认为这个领域应该有监管，但我们希望成为一个行动者，确保监管是外科手术，针对严重风险，并且是人们能够真正遵守的。

Because something I think the advocates of regulation don’t understand as well as they could is if we get something in place that’s poorly targeted, that wastes a bunch of people’s time, what’s going to happen is people are going to say, “See, these safety risks, this is nonsense.

因为我认为监管倡导者并没有充分理解，如果我们采取的措施针对性不强，浪费了很多人的时间，那么将会发生的事情是人们会说，“看，这些安全风险，这是无稽之谈。

I just had to hire 10 lawyers to fill out all these forms.

我只需要聘请 10 名律师来填写所有这些表格。

I had to run all these tests for something that was clearly not dangerous.”

我必须对显然不危险的东西进行所有这些测试。”

1:25:51 And after six months of that, there will be a ground swell and we’ll end up with a durable consensus against regulation.

六个月后，将会出现大幅上涨，我们最终将达成反对监管的持久共识。

And so I think the worst enemy of those who want real accountability is badly designed regulation.

因此，我认为那些想要真正问责的人最大的敌人是设计糟糕的监管。

We need to actually get it right.

我们需要真正做到正确。

And if there’s one thing I could say to the advocates, it would be that I want them to understand this dynamic better and we need to be really careful and we need to talk to people who actually have experience seeing how regulations play out in practice.

如果我可以对倡导者说一件事，那就是我希望他们更好地理解这种动态，我们需要非常小心，我们需要与那些真正有经验了解法规在实践中如何发挥作用的人交谈。

And the people who have seen that, understand to be very careful.

看到这一点的人都知道要非常小心。

If this was some lesser issue, I might be against regulation at all.

如果这是一个较小的问题，我可能会完全反对监管。

1:26:32 But what I want the opponents to understand is that the underlying issues are actually serious.

但我想让反对者明白的是，根本问题实际上很严重。

They’re not something that I or the other companies are just making up because of regulatory capture, they’re not sci-fi fantasies, they’re not any of these things.

它们不是我或其他公司因监管捕获而编造的东西，它们不是科幻幻想，它们不是任何这些东西。

Every time we have a new model, every few months we measure the behavior of these models and they’re getting better and better at these concerning tasks just as they are getting better and better at good, valuable, economically useful tasks.

每当我们有了一个新模型时，每隔几个月我们就会测量这些模型的行为，它们在这些相关任务上变得越来越好，就像它们在好的、有价值的、经济上有用的任务上变得越来越好一样。

And so I would just love it if some of the former, I think SB 1047 was very polarizing, I would love it if some of the most reasonable opponents and some of the most reasonable proponents would sit down together.

因此，如果前者，我认为 SB 1047 非常两极分化，我会喜欢它，如果一些最合理的反对者和一些最合理的支持者坐在一起，我会喜欢它。

And I think that the different AI companies, Anthropic was the only AI company that felt positively in a very detailed way.

我认为，在不同的人工智能公司中，Anthropic 是唯一一家在非常详细的方面表现出积极态度的人工智能公司。

I think Elon tweeted briefly something positive, but some of the big ones like Google, OpenAI, Meta, Microsoft were pretty staunchly against.

我认为埃隆在推特上简短地发布了一些积极的内容，但谷歌、OpenAI、Meta、微软等一些大公司却坚决反对。

1:27:49 So I would really is if some of the key stakeholders, some of the most thoughtful proponents and some of the most thoughtful opponents would sit down and say how do we solve this problem in a way that the proponents feel brings a real reduction in risk and that the opponents feel that it is not hampering the industry or hampering innovation any more necessary than it needs to.

所以我真的希望一些关键利益相关者、一些最有思想的支持者和一些最有思想的反对者能够坐下来讨论我们如何解决这个问题，让支持者认为可以真正降低风险，而反对者则认为这不会阻碍行业或阻碍创新。

I think for whatever reason, that things got too polarized and those two groups didn’t get to sit down in the way that they should.

我认为无论出于何种原因，事情变得过于两极分化，这两个群体没有以应有的方式坐下来。

And I feel urgency.

我感到紧迫。

I really think we need to do something in 2025.

我真的认为我们需要在 2025 年做点什么。

If we get to the end of 2025 and we’ve still done nothing about this, then I’m going to be worried.

如果到 2025 年底我们仍然没有采取任何行动，那么我会担心。

I’m not worried yet because, again, the risks aren’t here yet, but I think time is running short.

我还不担心，因为风险还没有到来，但我认为时间已经不多了。

And come up with something surgical, like you said.

就像你说的，想出一些外科手术的办法。

Yeah, yeah, yeah, exactly.

是啊是啊，没错。

1:28:46 And we need to get away from this intense pro safety versus intense anti-regulatory rhetoric.

我们需要摆脱这种强烈的支持安全与强烈的反监管言论。

It’s turned into these flame wars on Twitter and nothing good’s going to come of that.

Twitter 上的争论已经演变成激烈的争论，而且不会有任何好处。

So there’s a lot of curiosity about the different players in the game.

因此，人们对游戏中的不同玩家充满好奇。

1:29:04 One of the OGs is OpenAI.

OpenAI 就是其中之一。

You’ve had several years of experience at OpenAI.

您在 OpenAI 拥有多年的经验。

What’s your story and history there?

你在那里有什么故事和历史？

Yeah.

是的。

1:29:14 So I was at OpenAI for roughly five years.

我在 OpenAI 工作了大约五年。

For the last, I think it was couple years, I was vice president of research there.

最后，我想那是几年前的事了，我在那里担任研究副总裁。

Probably myself and Ilya Sutskever were the ones who really kind of set the research direction.

也许我和 Ilya Sutskever 才是真正确定研究方向的人。

Around 2016 or 2017, I first started to really believe in or at least confirm my belief in the scaling hypothesis when Ilya famously said to me, “The thing you need to understand about these models is they just want to learn.

大约在 2016 年或 2017 年，我第一次开始真正相信或至少证实了我对缩放假设的信念，当时 Ilya 对我说了一句著名的话：“关于这些模型，你需要了解的是，它们只是想学习。

The models just want to learn.” And again, sometimes there are these one sentences, these then cones, that you hear them and you’re like, “Ah, that explains everything.

模型只是想学习。”再说一次，有时会有这些句子，这些句子，你听到它们，你会想，“啊，这解释了一切。

That explains a thousand things that I’ve seen.” And then ever after, I had this visualization in my head of you optimize the models in the right way, you point the models in the right way, they just want to learn.

这解释了我所看到的一千件事。”从那时起，我的脑海中就出现了这样的可视化：你以正确的方式优化模型，你以正确的方式指向模型，他们只是想学习。

They just want to solve the problem regardless of what the problem is.

他们只是想解决问题，不管问题是什么。

So get out of their way, basically?

那么基本上就别挡他们的路了？

Get out of their way.

别挡他们的路。

1:30:10 Yeah.

是的。

Okay.

好的。

Don’t impose your own ideas about how they should learn.

不要将你自己的想法强加给他们应该如何学习。

1:30:11 And this was the same thing as Rich Sutton put out in the bitter lesson or Gwern put out in the scaling hypothesis.

这与里奇·萨顿（Rich Sutton）在惨痛教训中提出的观点或格温在缩放假设中提出的观点是一样的。

I think generally the dynamic was I got this kind of inspiration from Ilya and from others, folks like Alec Radford, who did the original GPT-1 and then ran really hard with it, me and my collaborators, on GPT-2, GPT-3, RL from Human Feedback, which was an attempt to kind of deal with the early safety and durability, things like debate and amplification, heavy on interpretability.

我认为一般来说，我从 Ilya 和其他人那里得到了这种灵感，比如 Alec Radford 等人，他们做了最初的 GPT-1，然后非常努力地使用它，我和我的合作者在 GPT-2、GPT-3、人类反馈的 RL 上得到了这种灵感，这是一种尝试处理早期的安全性和耐用性，比如辩论和放大，重在可解释性。

So again, the combination of safety plus scaling.

再次强调，安全性与扩展性的结合。

Probably 2018, 2019, 2020, those were kind of the years when myself and my collaborators, probably many of whom became co-founders of Anthropic, kind of really had a vision and drove the direction.

大概在 2018 年、2019 年、2020 年，那些年我和我的合作者（其中许多人可能成为了 Anthropic 的联合创始人）真正有了愿景并推动了方向。

Why’d you leave?

你为什么离开？

1:31:11 Why’d you decide to leave?

你为什么决定离开？

Yeah, so look, I’m going to put things this way and I think it ties to the race to the top, which is in my time at OpenAI, what I come to see as I’d come to appreciate the scaling hypothesis and as I’d come to appreciate kind of the importance of safety along with the scaling hypothesis.

是的，所以看，我打算这样说，我认为这与攀登顶峰的竞争有关，这是我在 OpenAI 的时候，当我开始欣赏扩展假设以及我开始欣赏安全性和扩展假设的重要性时，我所看到的。

1:31:13 The first one I think OpenAI was getting on board with.

我认为 OpenAI 正在参与的第一个项目。

The second one in a way had always been part of OpenAI’s messaging.

从某种程度上来说，第二个一直是 OpenAI 信息传递的一部分。

But over many years of the time that I spent there, I think I had a particular vision of how we should handle these things, how we should be brought out in the world, the kind of principles that the organization should have.

但在我在那里度过的多年时间里，我想我对我们应该如何处理这些事情、我们应该如何被带入世界、组织应该拥有什么样的原则有一个特殊的愿景。

And look, there were many, many discussions about should the company do this, should the company do that?

看，有很多很多关于公司应该这样做、应该那样做的讨论吗？

There’s a bunch of misinformation out there.

那里有很多错误信息。

1:32:07 People say we left because we didn’t like the deal with Microsoft.

人们说我们离开是因为我们不喜欢与微软的交易。

False.

错误的。

Although, it was like a lot of discussion, a lot of questions about exactly how we do the deal with Microsoft.

尽管如此，这就像很多讨论，很多关于我们如何与微软进行交易的问题。

We left because we didn’t like commercialization.

我们离开是因为我们不喜欢商业化。

That’s not true.

那不是真的。

We built GPD-3, which was the model that was commercialized.

我们建造了GPD-3，这是商业化的模型。

I was involved in commercialization.

我参与了商业化。

It’s more, again, about how do you do it?

更重要的是，你如何做到这一点？

Civilization is going down this path to very powerful AI.

文明正在沿着这条道路走向非常强大的人工智能。

What’s the way to do it?

有什么方法可以做到呢？

That is cautious, straightforward, honest, that builds trust in the organization and in individuals.

这是谨慎、坦率、诚实的，可以建立对组织和个人的信任。

How do we get from here to there and how do we have a real vision for how to get it right?

我们如何从这里到达那里？我们如何有一个真正的愿景来实现它？

How can safety not just be something we say because it helps with recruiting.

安全怎么能不仅仅是我们说的，因为它有助于招聘。

And I think at the end of the day, if you have a vision for that, forget about anyone else’s vision.

我认为归根结底，如果你有这样的愿景，那就忘记其他人的愿景。

1:33:01 I don’t want to talk about anyone else’s vision.

我不想谈论别人的愿景。

If you have a vision for how to do it, you should go off and you should do that vision.

如果你对如何去做有一个愿景，你应该开始行动，你应该实现这个愿景。

It is incredibly unproductive to try and argue with someone else’s vision.

试图与别人的愿景争论是极其没有成效的。

You might think they’re not doing it the right way.

您可能认为他们没有以正确的方式做事。

You might think they’re dishonest.

你可能认为他们不诚实。

Who knows?

谁知道？

Maybe you’re right, maybe you’re not.

也许你是对的，也许你不是。

But what you should do is you should take some people you trust and you should go off together and you should make your vision happen.

但你应该做的是，你应该带着一些你信任的人一起出发，你应该实现你的愿景。

And if your vision is compelling, if you can make it appeal to people, some combination of ethically in the market, if you can make a company that’s a place people want to join, that engages in practices that people think are reasonable while managing to maintain its position in the ecosystem at the same time, if you do that, people will copy it.

如果你的愿景引人注目，如果你能让它吸引人们，在市场上结合道德，如果你能让一家公司成为人们想要加入的地方，从事人们认为合理的做法，同时设法保持其在生态系统中的地位，如果你这样做，人们就会效仿它。

1:33:52 And the fact that you are doing it, especially the fact that you’re doing it better than they are, causes them to change their behavior in a much more compelling way than if they’re your boss and you’re arguing with them.

事实上，你正在这样做，尤其是你做得比他们更好，这会导致他们以一种比他们是你的老板而你与他们争论更引人注目的方式改变他们的行为。

I don’t know how to be any more specific about it than that, but I think it’s generally very unproductive to try and get someone else’s vision to look like your vision.

我不知道如何比这更具体，但我认为试图让别人的愿景与你的愿景相似通常是非常没有成效的。

It’s much more productive to go off and do a clean experiment and say, “This is our vision, this is how we’re going to do things.

开始做一个干净的实验并说：“这是我们的愿景，这就是我们做事的方式，这样会更有成效。

Your choice is you can ignore us, you can reject what we’re doing or you can start to become more like us.” And imitation is the sincerest form of flattery.

你的选择是你可以忽略我们，你可以拒绝我们正在做的事情，或者你可以开始变得更像我们。”而模仿是最真诚的奉承形式。

And that plays out in the behavior of customers, that plays out in the behavior of the public, that plays out in the behavior of where people choose to work.

这体现在客户的行为中，体现在公众的行为中，体现在人们选择工作地点的行为中。

And again, at the end, it’s not about one company winning or another company winning.

再说一遍，最终，这不是一家公司获胜或另一家公司获胜的问题。

1:34:48 If we or another company are engaging in some practice that people find genuinely appealing, and I want it to be in substance, not just an appearance and I think researchers are sophisticated and they look at substance, and then other companies start copying that practice and they win because they copied that practice.

如果我们或其他公司正在从事一些人们认为真正有吸引力的做法，并且我希望它具有实质内容，而不仅仅是外表，我认为研究人员很复杂，他们会关注实质内容，然后其他公司开始复制这种做法，他们就会获胜，因为他们复制了这种做法。

That’s great.

那太棒了。

That’s success.

这就是成功。

That’s like the race to the top.

这就像登顶的竞赛。

It doesn’t matter who wins in the end as long as everyone is copying everyone else’s good practices.

只要每个人都复制其他人的良好做法，最后谁获胜并不重要。

One way I think of it is the thing we’re all afraid of is the race to the bottom and the race to the bottom doesn’t matter who wins because we all lose.

我认为，我们都害怕的事情是逐底竞争，而逐底竞争并不重要谁赢，因为我们都输了。

In the most extreme world, we make this autonomous AI that the robots enslave us or whatever.

在最极端的世界里，我们制造了这种自主人工智能，让机器人奴役我们或其他什么。

That’s half joking, but that is the most extreme thing that could happen.

这是半开玩笑的，但这是可能发生的最极端的事情。

Then it doesn’t matter which company was ahead.

那么哪家公司领先并不重要。

If instead you create a race to the top where people are competing to engage in good practices, then at the end of the day, it doesn’t matter who ends up winning, it doesn’t even matter who started the race to the top.

相反，如果你创造了一场登顶竞赛，让人们竞相参与良好实践，那么最终，谁最终获胜并不重要，甚至谁开始了登顶竞赛也不重要。

1:35:57 The point isn’t to be virtuous, the point is to get the system into a better equilibrium than it was before.

重点不是要保持道德，重点是让系统达到比以前更好的平衡。

And individual companies can play some role in doing this.

个别公司可以在这方面发挥一些作用。

Individual companies can help to start it, can help to accelerate it.

个别公司可以帮助启动它，可以帮助加速它。

And frankly, I think individuals at other companies have done this as well.

坦率地说，我认为其他公司的个人也这样做了。

The individuals that when we put out an RSP react by pushing harder to get something similar done at other companies, sometimes other companies do something that’s we’re like, “Oh, it’s a good practice.

当我们推出RSP时，人们的反应是更加努力地推动其他公司完成类似的事情，有时其他公司会做一些我们喜欢的事情，“哦，这是一个很好的做法。

We think that’s good.

我们认为这很好。

We should adopt it too.” The only difference is I think we try to be more forward leaning.

我们也应该采纳它。”唯一的区别是我认为我们尝试更加前倾。

We try and adopt more of these practices first and adopt them more quickly when others invent them.

我们首先尝试并采用更多这些实践，并在其他人发明它们时更快地采用它们。

But I think this dynamic is what we should be pointing at and that I think it abstracts away the question of which company’s winning, who trusts who.

但我认为这种动态才是我们应该指出的，而且我认为它抽象了哪家公司获胜、谁信任谁的问题。

I think all these questions of drama are profoundly uninteresting and the thing that matters is the ecosystem that we all operate in and how to make that ecosystem better because that constrains all the players.

我认为所有这些戏剧性的问题都非常无趣，重要的是我们所有人都在其中运作的生态系统以及如何使该生态系统变得更好，因为这限制了所有参与者。

And so Anthropic is this kind of clean experiment built on a foundation of what concretely AI safety should look like?

那么 Anthropic 是这种干净的实验建立在具体的人工智能安全应该是什么样子的基础上吗？

Well, look, I’m sure we’ve made plenty of mistakes along the way.

好吧，看，我确信我们一路上犯了很多错误。

1:37:13 The perfect organization doesn’t exist.

完美的组织并不存在。

It has to deal with the imperfection of a thousand employees.

它必须应对一千名员工的不完美。

It has to deal with the imperfection of our leaders, including me.

它必须解决包括我在内的我们领导人的不完美问题。

It has to deal with the imperfection of the people we’ve put to oversee the imperfection of the leaders like the board and the long-term benefit trust.

它必须解决我们派来监督董事会和长期利益信托等领导者的不完美的问题。

It’s all a set of imperfect people trying to aim imperfectly at some ideal that will never perfectly be achieved.

这都是一群不完美的人试图不完美地瞄准某个永远无法完美实现的理想。

That’s what you sign up for.

这就是你注册的目的。

That’s what it will always be.

永远都是这样。

1:37:45 But imperfect doesn’t mean you just give up.

但不完美并不意味着你就放弃。

There’s better and there’s worse.

有更好的，也有更糟的。

And hopefully, we can do well enough that we can begin to build some practices that the whole industry engages in.

希望我们能够做得足够好，以便我们能够开始建立一些整个行业都参与的实践。

And then my guess is that multiple of these companies will be successful.

我的猜测是，其中多家公司将会取得成功。

Anthropic will be successful.

人择将会成功。

These other companies, like ones I’ve been at the past, will also be successful.

这些其他公司，就像我过去工作过的公司一样，也将取得成功。

And some will be more successful than others.

有些人会比其他人更成功。

That’s less important than, again, that we align the incentives of the industry.

再说一遍，这并不重要，重要的是我们调整行业的激励措施。

And that happens partly through the race to the top, partly through things like RSP, partly through, again, selected surgical regulation.

这部分是通过争先恐后、部分是通过 RSP 之类的事情、部分是通过选择性的外科手术监管来实现的。

You said talent density beats talent mass, so can you explain that?

您说人才密度胜过人才质量，您能解释一下吗？

1:38:25 Can you expand on that?

你能扩展一下吗？

Yeah.

是的。

Can you just talk about what it takes to build a great team of AI researchers and engineers?

您能谈谈如何打造一支优秀的人工智能研究人员和工程师团队吗？

This is one of these statements that’s more true every month.

这是每个月都更真实的陈述之一。

1:38:37 Every month I see this statement as more true than I did the month before.

每个月我都认为这句话比前一个月更真实。

So if I were to do a thought experiment, let’s say you have a team of 100 people that are super smart, motivated and aligned with the mission and that’s your company.

因此，如果我要做一个思想实验，假设您有一个由 100 人组成的团队，他们非常聪明、积极主动且与使命保持一致，这就是您的公司。

Or you can have a team of a thousand people where 200 people are super smart, super aligned with the mission and then 800 people are, let’s just say you pick 800 random big tech employees, which would you rather have?

或者你可以拥有一个 1000 人的团队，其中 200 人非常聪明，与使命高度一致，然后是 800 人，假设你随机挑选 800 名大型科技员工，你更愿意选择哪一个？

The talent mass is greater in the group of a thousand people.

千人群体中人才数量更大。

You have even a larger number of incredibly talented, incredibly aligned, incredibly smart people.

你甚至还有更多非常有才华、非常团结、非常聪明的人。

But the issue is just that if every time someone super talented looks around, they see someone else super talented and super dedicated, that sets the tone for everything.

但问题是，如果每次有超级才华横溢的人环顾四周时，他们都会看到其他人超级才华横溢且超级奉献，这就为一切定下了基调。

That sets the tone for everyone is super inspired to work at the same place.

这为每个人在同一个地方工作都受到极大的启发奠定了基调。

Everyone trusts everyone else.

每个人都信任其他人。

1:39:42 If you have a thousand or 10,000 people and things have really regressed, you are not able to do selection and you’re choosing random people, what happens is then you need to put a lot of processes and a lot of guardrails in place just because people don’t fully trust each other or you have to adjudicate political battles.

如果你有一千或一万人，事情确实倒退了，你无法进行选择，你只能随机选择人，那么你就需要建立很多流程和很多护栏，只是因为人们彼此不完全信任，或者你必须裁决政治斗争。

There are so many things that slow down the org’s ability to operate.

有很多因素会降低组织的运营能力。

And so we’re nearly a thousand people and we’ve tried to make it so that as large a fraction of those thousand people as possible are super talented, super skilled, it’s one of the reasons we’ve slowed down hiring a lot in the last few months.

所以我们有近千人，我们努力让这千人中尽可能多的人拥有超级才华、超级技能，这是我们在过去几个月放慢招聘速度的原因之一。

We grew from 300 to 800, I believe, I think in the first seven, eight months of the year and now we’ve slowed down.

我相信，今年前七、八个月，我们的人数从 300 人增加到了 800 人，现在我们已经放慢了速度。

The last three months, we went from 800 to 900, 950, something like that.

过去三个月，我们从 800 增加到 900、950，类似的情况。

Don’t quote me on the exact numbers, but I think there’s an inflection point around a thousand and we want to be much more careful how we grow.

不要引用我的确切数字，但我认为在一千左右有一个拐点，我们希望在增长方面更加谨慎。

1:40:42 Early on and now as well, we’ve hired a lot of physicists.

早期和现在，我们都聘请了很多物理学家。

Theoretical physicists can learn things really fast.

理论物理学家学东西的速度非常快。

Even more recently, as we’ve continued to hire that, we’ve really had a high bar on both the research side and the software engineering side, have hired a lot of senior people, including folks who used to be at other companies in this space, and we’ve just continued to be very selective.

甚至最近，随着我们继续招聘，我们在研究方面和软件工程方面确实有很高的标准，聘请了很多高级人才，包括曾经在这个领域的其他公司工作过的人，我们只是继续非常挑剔。

It’s very easy to go from a hundred to a thousand, a thousand to 10,000 without paying attention to making sure everyone has a unified purpose.

如果不注意确保每个人都有统一的目标，很容易从一百到一千，一千到一万。

It’s so powerful.

它是如此强大。

If your company consists of a lot of different fiefdoms that all want to do their own thing, they’re all optimizing for their own thing, it’s very hard to get anything done.

如果你的公司由很多不同的领域组成，每个领域都想做自己的事情，他们都在针对自己的事情进行优化，那么就很难完成任何事情。

But if everyone sees the broader purpose of the company, if there’s trust and there’s dedication to doing the right thing, that is a superpower.

但如果每个人都看到公司更广泛的目标，如果有信任并致力于做正确的事情，那就是一种超级力量。

That in itself I think can overcome almost every other disadvantage.

我认为这本身就可以克服几乎所有其他缺点。

And to Steve Jobs, A players.

还有史蒂夫·乔布斯，A级球员。

1:41:41 A players want to look around and see other A players is another way of saying that.

A 级玩家想要环顾四周并看到其他 A 级玩家是另一种表达方式。

Correct.

正确的。

I don’t know what that is about human nature, but it is demotivating to see people who are not obsessively driving towards a singular mission.

我不知道人性的本质是什么，但看到人们不执着地追求单一使命是令人沮丧的。

1:41:48 And it is on the flip side of that, super motivating to see that.

另一方面，看到这一点非常令人兴奋。

It’s interesting.

这很有趣。

What’s it take to be a great AI researcher or engineer from everything you’ve seen from working with so many amazing people?

从你与这么多了不起的人一起工作中所看到的一切来看，成为一名伟大的人工智能研究员或工程师需要什么？

Yeah.

是的。

1:42:09 I think the number one quality, especially on the research side, but really both, is open mindedness.

我认为最重要的品质，尤其是在研究方面，但实际上两者都是，是开放的思想。

Sounds easy to be open-minded, right?

听起来很容易保持开放态度，对吗？

You’re just like, “Oh, I’m open to anything.” But if I think about my own early history in this scaling hypothesis, I was seeing the same data others were seeing.

你就像，“哦，我对任何事情都持开放态度。”但如果我想想我自己在这个扩展假设中的早期历史，我看到的数据与其他人看到的数据相同。

I don’t think I was a better programmer or better at coming up with research ideas than any of the hundreds of people that I worked with.

我不认为我是一个更好的程序员，或者比我共事的数百人中的任何一个都更擅长提出研究想法。

In some ways, I was worse.

在某些方面，我更糟糕。

I’ve never precise programming of finding the bug, writing the GPU kernels.

我从来没有进行过精确的编程来查找错误、编写 GPU 内核。

I could point you to a hundred people here who are better at that than I am.

我可以向你指出这里有一百个人在这方面比我做得更好。

1:42:53 But the thing that I think I did have that was different was that I was just willing to look at something with new eyes.

但我认为我确实有一点不同，那就是我愿意用新的眼光来看待事物。

People said, “Oh, we don’t have the right algorithms yet.

人们说：“哦，我们还没有正确的算法。

We haven’t come up with the right way to do things.” And I was just like, “Oh, I don’t know.

我们还没有找到正确的做事方式。”我当时就想，“哦，我不知道。

This neural net has 30 million parameters.

这个神经网络有 3000 万个参数。

What if we gave it 50 million instead?

如果我们给它5000万呢？

Let’s plot some graphs.” That basic scientific mindset of like, “Oh man,” I see some variable that I could change.

让我们绘制一些图表。”那种基本的科学思维就像“天哪，”我看到了一些我可以改变的变量。

What happens when it changes?

当它改变时会发生什么？

Let’s try these different things and create a graph.

让我们尝试这些不同的事情并创建一个图表。

For even, this was the simplest thing in the world, change the number of, this wasn’t PhD level experimental design, this was simple and stupid.

甚至，这是世界上最简单的事情，改变数量，这不是博士级别的实验设计，这简单又愚蠢。

Anyone could have done this if you just told them that it was important.

如果你告诉他们这很重要，任何人都可以做到这一点。

It’s also not hard to understand.

这也不难理解。

You didn’t need to be brilliant to come up with this.

你不需要很聪明就能想出这个。

1:43:54 But you put the two things together and some tiny number of people, some single digit number of people have driven forward the whole field by realizing this.

但是你把这两件事放在一起，一些人，一些个位数的人通过意识到这一点推动了整个领域的发展。

And it’s often like that.

事情常常如此。

If you look back at the discoveries in history, they’re often like that.

如果你回顾历史上的发现，往往都是这样。

And so this open-mindedness and this willingness to see with new eyes that often comes from being newer to the field, often experience is a disadvantage for this, that is the most important thing.

因此，这种开放的思想和愿意用新的眼光看待事物的意愿往往来自于对这个领域的新手，经验往往是一个缺点，但这是最重要的。

It’s very hard to look for and test for, but I think it’s the most important thing because when you find something, some really new way of thinking about things, when you have the initiative to do that, it’s absolutely transformative.

寻找和测试是非常困难的，但我认为这是最重要的事情，因为当你发现一些东西，一些真正新的思考事物的方式，当你主动这样做时，这绝对是变革性的。

And also be able to do kind of rapid experimentation and, in the face of that, be open-minded and curious and looking at the data with these fresh eyes and seeing what is it that it’s actually saying.

并且还能够进行快速实验，面对这种情况，保持开放的心态和好奇心，用新的眼光看待数据，看看它实际上在说什么。

1:44:34 That applies in mechanistic interpretability.

这适用于机械可解释性。

It’s another example of this.

这是另一个例子。

1:44:46 Some of the early work and mechanistic interpretability so simple, it’s just no one thought to care about this question before.

一些早期的工作和机械解释是如此简单，只是之前没有人想到关心这个问题。

You said what it takes to be a great AI researcher.

你谈到了成为一名优秀的人工智能研究员需要什么。

1:44:56 Can we rewind the clock back, what advice would you give to people interested in AI?

时光倒流，您会给那些对人工智能感兴趣的人什么建议？

They’re young, looking…

他们很年轻，看起来……

1:45:00 What advice would you give to people interested in AI?

您会给那些对人工智能感兴趣的人什么建议？

They’re young.

他们很年轻。

Looking forward to how can I make an impact on the world?

期待如何对世界产生影响？

I think my number one piece of advice is to just start playing with the models.

我认为我的第一条建议是开始使用模型。

1:45:06 Actually, I worry a little, this seems like obvious advice now.

事实上，我有点担心，现在这似乎是显而易见的建议。

I think three years ago it wasn’t obvious and people started by, “Oh, let me read the latest reinforcement learning paper.” And you should do that as well, but now with wider availability of models and APIs, people are doing this more.

我认为三年前这并不明显，人们一开始就说：“哦，让我读一下最新的强化学习论文。”您也应该这样做，但现在随着模型和 API 的更广泛可用性，人们越来越多地这样做。

But, I think just experiential knowledge.

但是，我认为只是经验知识。

These models are new artifacts that no one really understands and so getting experience playing with them.

这些模型是新的工件，没有人真正理解，因此需要获得使用它们的经验。

I would also say again, in line with the do something new, think in some new direction, there are all these things that haven’t been explored.

我还要再说一遍，本着做一些新的事情，在一些新的方向上思考，所有这些东西都没有被探索过。

For example, mechanistic interpretability is still very new.

例如，机械可解释性仍然是非常新的。

It’s probably better to work on that than it is to work on new model architectures, because it’s more popular than it was before.

从事这一工作可能比研究新模型架构更好，因为它比以前更受欢迎。

There are probably 100 people working on it, but there aren’t like 10,000 people working on it.

可能有 100 个人在做这件事，但不会有 10,000 人在做这件事。

1:46:07 And it’s just this fertile area for study.

这正是研究的沃土。

There’s so much low-hanging fruit, you can just walk by and you can pick things.

有很多唾手可得的水果，你只需走过就可以摘东西。

For whatever reason, people aren’t interested in it enough.

无论出于何种原因，人们对此还不够感兴趣。

I think there are some things around long horizon learning and long horizon tasks, where there’s a lot to be done.

我认为，围绕长期学习和长期任务，有很多事情要做。

I think evaluations, we’re still very early in our ability to study evaluations, particularly for dynamic systems acting in the world.

我认为评估，我们研究评估的能力还处于早期阶段，特别是对于在世界上起作用的动态系统。

I think there’s some stuff around multi-agent.

我认为有一些关于多代理的东西。

Skate where the puck is going is my advice, and you don’t have to be brilliant to think of it.

我的建议是在冰球要去的地方滑行，你不需要很聪明就能想到这一点。

All the things that are going to be exciting in five years, people even mention them as conventional wisdom, but it’s just somehow there’s this barrier that people don’t double down as much as they could, or they’re afraid to do something that’s not the popular thing.

所有五年后将令人兴奋的事情，人们甚至将它们视为传统智慧，但不知何故存在这种障碍，人们不会尽其所能地加倍努力，或者他们害怕做一些不受欢迎的事情。

I don’t know why it happens, but getting over that barrier, that’s my number one piece of advice.

我不知道为什么会发生这种情况，但克服这个障碍，这是我的第一条建议。

Let’s talk if we could a bit about post-training.

我们可以谈谈训练后的情况吗？

1:47:14 So it seems that the modern post-training recipe has a little bit of everything.

所以看来现代的训练后食谱什么都有一点。

So supervised fine-tuning, RLHF, the constitutional AI with RLAIF-

所以有监督微调，RLHF，宪法人工智能与 RLAIF-

Best acronym.

最好的缩写。

It’s the, again, that naming thing.

这又是命名的问题。

1:47:33 And then synthetic data.

然后综合数据。

Seems like a lot of synthetic data, or at least trying to figure out ways to have high quality synthetic data.

似乎有很多合成数据，或者至少试图找出获得高质量合成数据的方法。

So if this is a secret sauce that makes Anthropic clause so incredible, how much of the magic is in the pre-training?

那么，如果这是使人择条款如此令人难以置信的秘密武器，那么预训练有多少魔力呢？

How much of it is in the post-training?

后期培训的内容有多少？

Yeah.

是的。

1:47:54 So first of all, we’re not perfectly able to measure that ourselves.

首先，我们自己无法完全衡量这一点。

When you see some great character ability, sometimes it’s hard to tell whether it came from pre-training or post-training.

当你看到一些出色的性格能力时，有时很难判断它是来自训练前还是训练后。

We developed ways to try and distinguish between those two, but they’re not perfect.

我们开发了一些方法来尝试区分这两者，但它们并不完美。

The second thing I would say is, when there is an advantage and I think we’ve been pretty good in general at RL, perhaps the best, although I don’t know, I don’t see what goes on inside other companies.

我要说的第二件事是，当有优势时，我认为我们在强化学习方面总体上表现得相当好，也许是最好的，尽管我不知道，我没有看到其他公司内部发生了什么。

Usually it isn’t, “Oh my God, we have this secret magic method that others don’t have.” Usually it’s like, “Well, we got better at the infrastructure so we could run it for longer,” or, “We were able to get higher quality data,” or, “We were able to filter our data better, or “We were able to combine these methods and practice.”

通常不是“天哪，我们有这种别人没有的秘法。”通常情况下，“我们在基础设施方面做得更好，因此我们可以运行更长时间”，或者“我们能够获得更高质量的数据”，或者“我们能够更好地过滤数据”，或者“我们能够将这些方法和实践结合起来。”

1:48:41 It’s usually some boring matter of practice and trade craft.

这通常是一些无聊的练习和技巧问题。

So when I think about how to do something special in terms of how we train these models both, but even more so I really think of it a little more, again, as designing airplanes or cars.

因此，当我思考如何在训练这些模型方面做一些特别的事情时，但更重要的是，我真的更多地想到了设计飞机或汽车。

It’s not just like, “Oh, man.

这不仅仅是“哦，伙计。

I have the blueprint.” Maybe that makes you make the next airplane.

我有蓝图。”也许这会让你制造下一架飞机。

But there’s some cultural trade craft of how we think about the design process that I think is more important than any particular gizmo we’re able to invent.

但我认为，我们如何看待设计过程的一些文化贸易技巧比我们能够发明的任何特定小玩意更重要。

Okay.

好的。

1:49:17 Well, let me ask you about specific techniques.

那我就请教一下具体的技巧吧。

So first on RLHF, what do you think, just zooming out intuition, almost philosophy …

首先，关于 RLHF，你觉得怎么样，只是缩小了直觉，几乎是哲学……

Why do you think RLHF works so well?

您认为 RLHF 为什么效果这么好？

If I go back to the scaling hypothesis, one of the ways to skate the scaling hypothesis is, if you train for X and you throw enough compute at it, then you get X.

如果我回到缩放假设，滑过缩放假设的方法之一是，如果你训练 X 并且对其进行足够的计算，那么你就会得到 X。

1:49:28 And so RLHF is good at doing what humans want the model to do, or at least to state it more precisely doing what humans who look at the model for a brief period of time and consider different possible responses, what they prefer as the response, which is not perfect from both the safety and capabilities perspective, in that humans are often not able to perfectly identify what the model wants and what humans want in the moment may not be what they want in the long term.

因此，RLHF 擅长做人类希望模型做的事情，或者至少更准确地说明它做的是人类在短时间内观察模型并考虑不同可能的反应，他们更喜欢的反应，从安全和能力的角度来看，这并不完美，因为人类往往无法完美地识别模型想要什么，而人类目前想要的可能不是他们长期想要的。

1:50:05 So there’s a lot of subtlety there, but the models are good at producing what the humans in some shallow sense want.

所以这里面有很多微妙之处，但这些模型擅长产生人类在某种浅层意义上想要的东西。

And it actually turns out that you don’t even have to throw that much compute at it, because of another thing, which is this thing about a strong pre-trained model being halfway to anywhere.

事实上，你甚至不需要投入那么多的计算，因为另一件事，那就是一个强大的预训练模型已经走到了任何地方。

So once you have the pre-trained model, you have all the representations you need to get the model where you want it to go.

因此，一旦有了预训练的模型，您就拥有了将模型带到您想要的位置所需的所有表示。

So do you think RLHF makes the model smarter, or just appear smarter to the humans?

那么您认为 RLHF 是让模型变得更聪明，还是只是让人类看起来更聪明？

I don’t think it makes the model smarter.

我不认为这会让模型变得更聪明。

1:50:41 I don’t think it just makes the model appear smarter.

我不认为这只是让模型显得更聪明。

It’s like RLHF bridges the gap between the human and the model.

这就像 RLHF 架起了人类和模型之间的桥梁。

I could have something really smart that can’t communicate at all.

我可以拥有一些非常聪明但根本无法交流的东西。

We all know people like this, people who are really smart but can’t understand what they’re saying.

我们都认识这样的人，他们非常聪明，但听不懂他们在说什么。

So I think RLHF just bridges that gap.

所以我认为 RLHF 正好弥补了这一差距。

I think it’s not the only kind of RL we do.

我认为这不是我们所做的唯一一种强化学习。

It’s not the only kind of RL that will happen in the future.

这并不是未来唯一一种强化学习。

I think RL has the potential to make models smarter, to make them reason better, to make them operate better, to make them develop new skills even.

我认为强化学习有潜力让模型变得更聪明，让它们更好地推理，让它们更好地运作，甚至让它们发展新技能。

And perhaps that could be done even in some cases with human feedback.

也许在某些情况下，通过人类反馈也可以做到这一点。

But, the kind of RLHF we do today mostly doesn’t do that yet, although we’re very quickly starting to be able to.

但是，我们今天所做的 RLHF 大多还没有做到这一点，尽管我们很快就开始能够做到这一点。

But if you look at the metric of helpfulness, it increases that?

但如果你看看有用性的指标，它会增加吗？

Yes.

是的。

1:51:36 It also increases, what was this word in Leopold’s essay, “unhobbling,” where basically the models are hobbled and then you do various trainings to them to unhobble them.

它也增加了，利奥波德的文章中的这个词是什么，“unhobbling”，基本上模型是被束缚的，然后你对它们进行各种训练以解除它们的束缚。

So I like that word, because it’s a rare word.

所以我喜欢这个词，因为它是一个罕见的词。

So I think RLHF unhobbles the models in some ways.

所以我认为 RLHF 在某些方面解除了模型的障碍。

And then there are other ways where that model hasn’t yet been unhobbled and needs to unhobble.

还有其他一些方法使该模型尚未摆脱束缚并且需要摆脱束缚。

If you can say in terms of cost, is pre-training the most expensive thing?

如果从成本上来说，预训练是不是最贵的？

1:51:58 Or is post-training creep up to that?

或者说训练后的表现已经达到了这个程度？

At the present moment, it is still the case that pre-training is the majority of the cost.

目前，预训练仍然是成本的主要部分。

1:52:05 I don’t know what to expect in the future, but I could certainly anticipate a future where post-training is the majority of the cost.

我不知道未来会发生什么，但我当然可以预见未来培训后将占据大部分成本。

In that future you anticipate, would it be the humans or the AI that’s the costly thing for the post-training?

在您预计的未来，是人类还是人工智能才是后期培训成本最高的部分？

I don’t think you can scale up humans enough to get high quality.

我认为你无法将人类规模扩大到足以获得高质量。

1:52:22 Any kind of method that relies on humans and uses a large amount of compute, it’s going to have to rely on some scaled supervision method, like debate or iterated amplification or something like that.

任何依赖人类并使用大量计算的方法，都必须依赖某种规模化的监督方法，例如辩论或迭代放大或类似的方法。

So on that super interesting set of ideas around constitutional AI, can you describe what it is as first detailed in December 2022 paper and beyond that.

那么，关于宪法人工智能的一系列超级有趣的想法，你能描述一下 2022 年 12 月论文中首次详细介绍的内容以及其他内容吗？

1:52:39 What is it?

它是什么？

Yes.

是的。

1:52:53 So this was from two years ago.

所以这是两年前的事了。

The basic idea is, so we describe what RLHF is.

基本思想是，所以我们描述什么是RLHF。

You have a model and you just sample from it twice.

你有一个模型，你只需从中采样两次。

It spits out two possible responses, and you’re like, “Human, which responses do you like better?” Or another variant of it is, “Rate this response on a scale of one to seven.” So that’s hard because you need to scale up human interaction and it’s very implicit.

它会给出两种可能的反应，你会说，“人类，你更喜欢哪种反应？”或者它的另一种变体是，“按照从 1 到 7 的等级对这个响应进行评分。”所以这很难，因为你需要扩大人际互动，而且这是非常隐性的。

I don’t have a sense of what I want the model to do.

我不知道我想要模型做什么。

I just have a sense of what this average of 1,000 humans wants the model to do.

我只是知道这 1000 个人平均希望模型做什么。

So two ideas.

所以有两个想法。

One is, could the AI system itself decide which response is better?

一是，人工智能系统本身能否决定哪种反应更好？

Could you show the AI system these two responses and ask which response is better?

你能否向人工智能系统展示这两种反应，并询问哪种反应更好？

And then second, well, what criterion should the AI use?

其次，人工智能应该使用什么标准？

1:53:43 And so then there’s this idea, you have a single document, a constitution if you will, that says, these are the principles the model should be using to respond.

所以就有了这个想法，如果你愿意的话，你有一个单一的文件，一个宪法，它说，这些是模型应该用来响应的原则。

And the AI system reads those reads principles as well as reading the environment and the response.

人工智能系统会读取这些读取原理以及读取环境和响应。

And it says, “Well, how good did the AI model do?” It’s basically a form of self-play.

它说：“那么，人工智能模型表现如何？”这基本上是一种自我游戏的形式。

You’re training the model against itself.

您正在针对模型本身进行训练。

And so the AI gives the response and then you feed that back into what’s called the preference model, which in turn feeds the model to make it better.

因此，人工智能会给出响应，然后将其反馈到所谓的偏好模型中，偏好模型又会反馈给模型，使其变得更好。

So you have this triangle of the AI, the preference model, and the improvement of the AI itself.

所以你就有了人工智能、偏好模型和人工智能本身的改进的三角关系。

And we should say that in the constitution, the set of principles are human interpretable.

我们应该说，在宪法中，这套原则是人类可以解释的。

1:54:22 They’re-

他们是——

Yeah.

是的。

1:54:27 Yeah.

是的。

It’s something both the human and the AI system can read.

这是人类和人工智能系统都可以读取的内容。

So it has this nice translatability or symmetry.

所以它具有很好的可翻译性或对称性。

In practice, we both use a model constitution and we use RLHF and we use some of these other methods.

在实践中，我们既使用模型宪法，又使用 RLHF，还使用其中一些其他方法。

So it’s turned into one tool in a toolkit, that both reduces the need for RLHF and increases the value we get from using each data point of RLHF.

因此，它变成了工具包中的一个工具，既减少了对 RLHF 的需求，又增加了我们从使用 RLHF 的每个数据点中获得的价值。

It also interacts in interesting ways with future reasoning type RL methods.

它还以有趣的方式与未来推理类型的 RL 方法进行交互。

So it’s one tool in the toolkit, but I think it is a very important tool.

所以它是工具包中的一个工具，但我认为它是一个非常重要的工具。

Well, it’s a compelling one to us humans.

嗯，这对我们人类来说是一个引人注目的问题。

1:55:05 Thinking about the founding fathers and the founding of the United States.

思考开国元勋和美国的建国历程。

The natural question is who and how do you think it gets to define the constitution, the set of principles in the constitution?

自然的问题是，你认为谁以及如何定义宪法，宪法中的一套原则？

Yeah.

是的。

1:55:20 So I’ll give a practical answer and a more abstract answer.

所以我会给出一个实际的答案和一个更抽象的答案。

I think the practical answer is look in practice, models get used by all kinds of different customers.

我认为实际的答案是在实践中观察，模型被各种不同的客户使用。

And so you can have this idea where the model can have specialized rules or principles.

因此，您可以想到模型可以具有专门的规则或原则。

We fine tune versions of models implicitly.

我们隐式地微调模型的版本。

We’ve talked about doing it explicitly having special principles that people can build into the models.

我们已经讨论过要明确地制定特殊原则，人们可以将这些原则构建到模型中。

So from a practical perspective, the answer can be very different from different people.

因此，从实际角度来看，不同的人的答案可能有很大不同。

A customer service agent behaves very differently from a lawyer and obeys different principles.

客户服务代理的行为与律师非常不同，遵循不同的原则。

1:55:57 But, I think at the base of it, there are specific principles that models have to obey.

但是，我认为在其基础上，模型必须遵守一些特定的原则。

I think a lot of them are things that people would agree with.

我认为其中很多都是人们会同意的事情。

Everyone agrees that we don’t want models to present these CBRN risks.

每个人都同意我们不希望模型呈现这些 CBRN 风险。

I think we can go a little further and agree with some basic principles of democracy and the rule of law.

我认为我们可以更进一步，就民主和法治的一些基本原则达成一致。

Beyond that, it gets very uncertain and there our goal is generally for the models to be more neutral, to not espouse a particular point of view and more just be wise agents or advisors that will help you think things through and will present possible considerations.

除此之外，它变得非常不确定，我们的目标通常是让模型更加中立，不支持特定的观点，而更多地只是成为明智的代理人或顾问，帮助您思考问题并提出可能的考虑因素。

But don’t express strong or specific opinions.

但不要表达强烈或具体的意见。

OpenAI released a model spec where it clearly, concretely defines some of the goals of the model and specific examples like AB, how the model should behave.

OpenAI 发布了一个模型规范，其中清楚、具体地定义了模型的一些目标和具体示例，例如 AB，模型应该如何表现。

1:56:42 Do you find that interesting?

你觉得有趣吗？

By the way I should mention, I believe the brilliant John Schulman was a part of that.

顺便说一句，我相信才华横溢的约翰·舒尔曼也是其中的一部分。

He’s now at Anthropic.

他现在在 Anthropic。

Do you think this is a useful direction?

您认为这是一个有用的方向吗？

Might Anthropic release a model spec as well?

Anthropic 也可能发布模型规格吗？

Yeah.

是的。

1:57:05 So I think that’s a pretty useful direction.

所以我认为这是一个非常有用的方向。

Again, it has a lot in common with constitutional AI.

同样，它与宪法人工智能有很多共同点。

So again, another example of a race to the top.

这又是一个力争上游的例子。

We have something that we think a better and more responsible way of doing things.

我们有一些我们认为更好、更负责任的做事方式。

It’s also a competitive advantage.

这也是一种竞争优势。

Then others discover that it has advantages and then start to do that thing.

然后其他人发现它有优点，然后开始做那件事。

We then no longer have the competitive advantage, but it’s good from the perspective that now everyone has adopted a positive practice that others were not adopting.

这样我们就不再拥有竞争优势，但从现在每个人都采取了其他人没有采用的积极做法的角度来看，这是件好事。

And so our response to that is, “Well, looks like we need a new competitive advantage in order to keep driving this race upwards.” So that’s how I generally feel about that.

因此，我们对此的回应是：“好吧，看来我们需要新的竞争优势，才能继续推动这场竞赛的发展。”这就是我对此的总体感受。

I also think every implementation of these things is different.

我还认为这些事情的每次实施都是不同的。

So there were some things in the model spec that were not in constitutional AI, and so we can always adopt those things or at least learn from them.

因此，模型规范中有些东西不属于宪法人工智能，所以我们总是可以采用这些东西，或者至少从中学习。

So again, I think this is an example of the positive dynamic that I think we should all want the field to have.

再说一次，我认为这是一个积极动态的例子，我认为我们都应该希望这个领域拥有这种积极动态。

Let’s talk about the incredible essay Machines of Loving Grace.

让我们来谈谈令人难以置信的文章《爱的机器》。

1:58:06 I recommend everybody read it.

我建议大家阅读它。

It’s a long one.

这是一篇很长的文章。

It is rather long.

它相当长。

Yeah.

是的。

1:58:13 It’s really refreshing to read concrete ideas about what a positive future looks like.

阅读有关积极未来的具体想法确实令人耳目一新。

And you took a bold stance because it’s very possible that you might be wrong on the dates or the specific applications-

你采取了大胆的立场，因为你很可能在日期或具体应用上是错误的——

Oh, yeah.

哦，是的。

1:58:24 I’m fully expecting to well, definitely be wrong about all the details.

我完全预计所有细节肯定都是错误的。

I might be just spectacularly wrong about the whole thing and people will laugh at me for years.

我可能在整件事上都错得离谱，人们会嘲笑我很多年。

That’s just how the future works.

这就是未来的运作方式。

So you provided a bunch of concrete positive impacts of AI and how exactly a super intelligent AI might accelerate the rate of breakthroughs in, for example, biology and chemistry, that would then lead to things like we cure most cancers, prevent all infectious disease, double the human lifespan and so on.

因此，您提供了人工智能的一系列具体的积极影响，以及超级智能人工智能如何加速生物学和化学等领域的突破速度，从而导致我们治愈大多数癌症、预防所有传染病、使人类寿命加倍等等。

1:58:40 So let’s talk about this essay first.

那么我们先来谈谈这篇文章。

Can you give a high-level vision of this essay?

你能对这篇文章给出一个高层次的展望吗？

And what are the key takeaways that people have?

人们的主要收获是什么？

Yeah.

是的。

1:59:08 I have spent a lot of time, and in Anthropic has spent a lot of effort on how do we address the risks of AI?

我花了很多时间，在Anthropic上花了很多精力来研究我们如何应对AI的风险？

How do we think about those risks?

我们如何看待这些风险？

We’re trying to do a race to the top, what that requires us to build all these capabilities and the capabilities are cool.

我们正在努力争夺冠军，这需要我们建立所有这些能力，而且这些能力很酷。

But, a big part of what we’re trying to do is address the risks.

但是，我们正在努力做的很大一部分是解决风险。

And the justification for that is like, well, all these positive things, the market is this very healthy organism.

其理由是，所有这些积极的事情，市场是一个非常健康的有机体。

It’s going to produce all the positive things.

它将产生所有积极的事情。

The risks?

风险？

I don’t know, we might mitigate them, we might not.

我不知道，我们可能会减轻它们，也可能不会。

And so we can have more impact by trying to mitigate the risks.

因此，我们可以通过努力降低风险来产生更大的影响。

1:59:46 But, I noticed that one flaw in that way of thinking, and it’s not a change in how seriously I take the risks.

但是，我注意到这种思维方式的一个缺陷，但这并没有改变我对待风险的认真程度。

It’s maybe a change in how I talk about them, is that no matter how logical or rational, that line of reasoning that I just gave might be.

也许是我谈论它们的方式发生了变化，无论我刚才给出的推理有多合乎逻辑或合理。

If you only talk about risks, your brain only thinks about risks.

如果你只谈论风险，你的大脑只会考虑风险。

And so, I think it’s actually very important to understand, what if things do go well?

所以，我认为理解这一点实际上非常重要：如果事情进展顺利怎么办？

And the whole reason we’re trying to prevent these risks is not because we’re afraid of technology, not because we want to slow it down.

我们试图预防这些风险的全部原因并不是因为我们害怕技术，也不是因为我们想放慢它的速度。

It’s because if we can get to the other side of these risks, if we can run the gauntlet successfully, to put it in stark terms, then on the other side of the gauntlet are all these great things.

这是因为，如果我们能够克服这些风险，如果我们能够成功地应对挑战，用严格的术语来说，那么挑战的另一面就是所有这些伟大的事情。

2:00:36 And these things are worth fighting for.

这些事情都值得为之奋斗。

And these things can really inspire people.

这些事情确实可以激励人们。

And I think I imagine, because …

我想我想象的是，因为……

Look, you have all these investors, all these VCs, all these AI companies talking about all the positive benefits of AI.

看，所有这些投资者、所有这些风险投资家、所有这些人工智能公司都在谈论人工智能的所有积极好处。

But as you point out, it’s weird.

但正如你所指出的，这很奇怪。

There’s actually a dearth of really getting specific about it.

实际上缺乏真正具体的说明。

There’s a lot of random people on Twitter posting these gleaming cities and this just vibe of grind, accelerate harder, kick out the …

Twitter 上有很多随机的人发布了这些闪闪发光的城市，这只是一种磨砺的氛围，更努力地加速，踢出……

It’s just this very aggressive ideological.

这就是非常激进的意识形态。

But then you’re like, “Well, what are you actually excited about?”

但你会想，“好吧，你到底对什么感到兴奋？”

2:01:17 And so, I figured that I think it would be interesting and valuable for someone who’s actually coming from the risk side to try and really make a try at explaining what the benefits are, both because I think it’s something we can all get behind and I want people to understand.

因此，我认为对于真正来自风险方面的人来说，尝试并真正尝试解释好处是什么，这将是有趣且有价值的，因为我认为这是我们所有人都可以支持的事情，而且我希望人们能够理解。

I want them to really understand that this isn’t Doomers versus Accelerationists.

我希望他们真正明白，这不是末日论者与加速论者之间的较量。

This is that, if you have a true understanding of where things are going with AI, and maybe that’s the more important axis, AI is moving fast versus AI is not moving fast, then you really appreciate the benefits and you really want humanity or civilization to seize those benefits.

也就是说，如果你真正了解人工智能的发展方向，也许这是更重要的轴，人工智能正在快速发展，而人工智能却没有快速发展，那么你就会真正体会到其中的好处，并且你真的希望人类或文明能够抓住这些好处。

But, you also get very serious about anything that could derail them.

但是，你也会非常认真地对待任何可能使他们脱轨的事情。

So I think the starting point is to talk about what this Powerful AI, which is the term you like to use, most of the world uses AGI, but you don’t like the term, because it’s basically has too much baggage, it’s become meaningless.

所以我认为首先要谈谈这个强大的人工智能是什么，这是你喜欢使用的术语，世界上大多数人都使用AGI，但你不喜欢这个术语，因为它基本上有太多的包袱，它变得毫无意义。

2:02:09 It’s like we’re stuck with the terms whether we like them or not.

就好像无论我们喜欢与否，我们都被这些术语所束缚。

Maybe we’re stuck with the terms and my efforts to change them are futile.

也许我们被这些条款所束缚，而我改变它们的努力是徒劳的。

It’s admirable.

令人钦佩。

I’ll tell you what else I don’t …

我会告诉你还有什么我不知道的……

2:02:29 This is a pointless semantic point, but I keep talking about it-

这是一个毫无意义的语义点，但我一直在谈论它——

It’s back to naming again.

又回到命名了。

I’m just going to do it once more.

我只想再做一次。

2:02:36 I think it’s a little like, let’s say it was like 1995 and Moore’s law is making the computers faster.

我认为这有点像 1995 年，摩尔定律让计算机变得更快。

And for some reason there had been this verbal tick that everyone was like, “Well, someday we’re going to have supercomputers.

出于某种原因，每个人都在口头上嘀咕：“好吧，有一天我们将会拥有超级计算机。

And supercomputers are going to be able to do all these things that …

超级计算机将能够完成所有这些事情……

Once we have supercomputers, we’ll be able to sequence the genome, we’ll be able to do other things.” And so.

一旦我们拥有超级计算机，我们将能够对基因组进行测序，我们将能够做其他事情。”等等。

One, it’s true, the computers are getting faster and as they get faster, they’re going to be able to do all these great things.

第一，确实，计算机变得越来越快，随着它们变得越来越快，它们将能够完成所有这些伟大的事情。

But there’s, there’s no discrete point at which you had a supercomputer and previous computers were no.

但是，没有一个离散点可以证明你拥有一台超级计算机，而以前的计算机则没有。

“Supercomputer” is a term we use, but it’s a vague term to just describe computers that are faster than what we have today.

“超级计算机”是我们使用的一个术语，但它是一个模糊的术语，仅描述比我们今天拥有的计算机更快的计算机。

2:03:19 There’s no point at which you pass the threshold and you’re like, “Oh, my God!

当你跨过门槛时，你不会说：“哦，天哪！

We’re doing a totally new type of computation and new …

我们正在做一种全新的计算类型和新的……

And so I feel that way about AGI.

所以我对 AGI 也有这样的感觉。

There’s just a smooth exponential.

只有一个平滑的指数。

And if by AGI you mean AI is getting better and better, and gradually it’s going to do more and more of what humans do until it’s going to be smarter than humans, and then it’s going to get smarter even from there, then yes, I believe in AGI.

如果你所说的 AGI 是指人工智能变得越来越好，并且逐渐地它会做越来越多的人类所做的事情，直到它比人类更聪明，然后甚至从那时起它就会变得更聪明，那么是的，我相信 AGI。

But, if AGI is some discrete or separate thing, which is the way people often talk about it, then it’s a meaningless buzzword.

但是，如果通用人工智能是一些离散或独立的事物，就像人们经常谈论它的方式一样，那么它就是一个毫无意义的流行词。

To me, it’s just a platonic form of a powerful AI, exactly how you define it.

对我来说，这只是强大人工智能的柏拉图式形式，正如你所定义的那样。

2:03:50 You define it very nicely, so on the intelligence axis, it’s just on pure intelligence, it’s smarter than a Nobel Prize winner as you describe across most relevant disciplines.

你对它的定义非常好，所以在智力轴上，它只是纯粹的智力，正如你在大多数相关学科中所描述的那样，它比诺贝尔奖获得者还要聪明。

So okay, that’s just intelligence.

好吧，这只是智力。

So it’s both in creativity and be able to generate new ideas, all that kind of stuff in every discipline, Nobel Prize winner in their prime.

因此，它既具有创造力，又能够产生新的想法，每个学科都有这样的东西，诺贝尔奖获得者正值巅峰。

It can use every modality, so this is self-explanatory, but just operate across all the modalities of the world.

它可以使用每种模式，所以这是不言自明的，但只是在世界上所有模式中运行。

2:04:28 It can go off for many hours, days and weeks to do tasks and do its own detailed planning and only ask you help when it’s needed.

它可以花费数小时、数天和数周的时间来完成任务并制定自己的详细计划，并且仅在需要时才向您寻求帮助。

This is actually interesting.

这其实很有趣。

I think in the essay you said …

我认为你在文章中说...

Again, it’s a bet that it’s not going to be embodied, but it can control embodied tools.

再次强调，它不会被具体化，但它可以控制具体化的工具。

So it can control tools, robots, laboratory equipment., the resource used to train it can then be repurposed to run millions of copies of it, and each of those copies would be independent that could do their own independent work.

因此它可以控制工具、机器人、实验室设备。然后，用于训练它的资源可以重新用于运行它的数百万个副本，并且每个副本都是独立的，可以完成自己独立的工作。

So you can do the cloning of the intelligence systems.

所以你可以克隆智能系统。

Yeah.

是的。

2:05:03 Yeah.

是的。

You might imagine from outside the field that there’s only one of these, right?

从场外你可能会想象只有其中之一，对吗？

You’ve only made one.

你只做了一个。

But the truth is that the scale up is very quick.

但事实是，规模扩大的速度非常快。

We do this today,.

我们今天就这样做。

We make a model, and then we deploy thousands, maybe tens of thousands of instances of it.

我们创建一个模型，然后部署数千个甚至数万个它的实例。

I think by the time, certainly within two to three years, whether we have these super powerful AIs or not, clusters are going to get to the size where you’ll be able to deploy millions of these.

我认为，到那时，当然是在两到三年内，无论我们是否拥有这些超级强大的人工智能，集群都将达到能够部署数百万个的规模。

And they’ll be faster than humans.

而且它们会比人类更快。

And so, if your picture is, “Oh, we’ll have one and it’ll take a while to make them,” my point there was, no.

所以，如果你的照片是，“哦，我们会有一个，需要一段时间才能制作出来”，我的观点是，不。

Actually you have millions of them right away.

事实上你马上就拥有了数百万个。

And in general they can learn and act 10 to 100 times faster than humans.

一般来说，它们的学习和行动速度比人类快 10 到 100 倍。

2:05:37 So that’s a really nice definition of powerful AI.

所以这是对强大人工智能的一个非常好的定义。

Okay, so that.

好吧，就这样吧。

But, you also write that, “Clearly such an entity would be capable of solving very difficult problems very fast, but it is not trivial to figure out how fast.

但是，您还写道，“显然这样的实体能够非常快地解决非常困难的问题，但弄清楚有多快并不是一件容易的事。

Two “extreme” positions both seem false to me.” So the singularity is on the one extreme and the opposite and the other extreme.

在我看来，两种“极端”立场都是错误的。”因此，奇点位于一个极端，而相反，又是另一个极端。

Can you describe each of the extremes?

您能描述一下每一个极端情况吗？

Yeah.

是的。

So why?

那么为什么呢？

So yeah.

所以是的。

2:06:06 Let’s describe the extreme.

我们来描述一下极端情况。

So one extreme would be, “Well, look.

所以一个极端是，“好吧，看。

If we look at evolutionary history like there was this big acceleration, where for hundreds of thousands of years we just had single-celled organisms, and then we had mammals, and then we had apes.

如果我们回顾一下进化史，就像有一个巨大的加速，数十万年来我们只有单细胞生物，然后我们有了哺乳动物，然后我们有了猿。

And then that quickly turned to humans.

然后很快就转向人类。

Humans quickly built industrial civilization.” And so, this is going to keep speeding up and there’s no ceiling at the human level.

人类迅速建立了工业文明。”因此，这将继续加速，并且人类水平没有上限。

Once models get much, much smarter than humans, they’ll get really good at building the next models.

一旦模型变得比人类聪明得多，他们就会非常擅长构建下一个模型。

And if you write down a simple differential equation, like this is an exponential …

如果你写下一个简单的微分方程，就像这是一个指数......

And so what’s going to happen is that models will build faster models.

因此，将会发生的情况是模型将构建更快的模型。

Models will build faster models.

模型将构建更快的模型。

And those models will build nanobots that can take over the world and produce much more energy than you could produce otherwise.

这些模型将构建出可以接管世界并产生比其他方式更多的能量的纳米机器人。

And so, if you just kind of solve this abstract differential equation, then like five days after we build the first AI that’s more powerful than humans, then the world will be filled with these AIs in every possible technology that could be invented, like will be invented.

因此，如果你只是解决这个抽象的微分方程，那么就像我们构建第一个比人类更强大的人工智能五天后，世界上将充满这些人工智能，在每一种可能发明的技术中，就像将要发明的那样。

2:07:12 I’m caricaturing this a little bit, but I think that’s one extreme.

我有点讽刺这一点，但我认为这是一个极端。

And the reason that I think that’s not the case is that, one, I think they just neglect the laws of physics.

我认为情况并非如此的原因是，第一，我认为他们只是忽视了物理定律。

It’s only possible to do things so fast in the physical world.

只有在物质世界里才能做到如此快的事情。

Some of those loops go through producing faster hardware.

其中一些循环会产生更快的硬件。

It takes a long time to produce faster hardware.

生产更快的硬件需要很长时间。

Things take a long time.

事情需要很长时间。

There’s this issue of complexity.

这里存在一个复杂性问题。

I think no matter how smart you are, people talk about, “Oh, we can make models of biological systems that’ll do everything the biological systems …

我认为无论你多么聪明，人们都会谈论，“哦，我们可以制作生物系统模型，它可以完成生物系统的所有事情......

” Look, I think computational modeling can do a lot.

” 看，我认为计算建模可以做很多事情。

I did a lot of computational modeling when I worked in biology.

当我从事生物学工作时，我做了很多计算建模。

But just there are a lot of things that you can’t predict how …

但很多事情是你无法预测的……

They’re complex enough that just iterating, just running the experiment is going to beat any modeling, no matter how smart the system doing the modeling is.

它们足够复杂，只要迭代、运行实验就可以击败任何建模，无论进行建模的系统有多智能。

Well, even if it’s not interacting with the physical world, just the modeling is going to be hard?

好吧，即使不与物理世界交互，仅仅建模就会很困难？

Yeah.

是的。

2:08:12 Well, the modeling is going to be hard and getting the model to match the physical world is going to be

嗯，建模将会很困难，并且让模型与物理世界相匹配将会很困难

All right.

好吧。

2:08:18 So it does have to interact with the physical world to verify.

所以它确实必须与物理世界进行交互来验证。

But you just look at even the simplest problems.

但你只看最简单的问题。

2:08:21 I think I talk about The Three-Body Problem or simple chaotic prediction, or predicting the economy.

我想我谈论的是三体问题或简单的混沌预测，或预测经济。

It’s really hard to predict the economy two years out.

预测两年后的经济确实很难。

Maybe the case is humans can predict what’s going to happen in the economy next quarter, or they can’t really do that.

也许情况是人类可以预测下个季度经济将发生什么，或者他们不能真正做到这一点。

Maybe a AI that’s a zillion times smarter can only predict it out a year or something, instead of …

也许一个聪明无数倍的人工智能只能预测一年之类的事情，而不是……

You have these exponential increase in computer intelligence for linear increase in ability to predict.

计算机智能呈指数级增长，预测能力呈线性增长。

Same with again, like biological molecules interacting.

同样，就像生物分子相互作用一样。

You don’t know what’s going to happen when you perturb a complex system.

当你扰乱一个复杂的系统时，你不知道会发生什么。

You can find simple parts in it, if you’re smarter, you’re better at finding these simple parts.

你可以在里面找到简单的零件，如果你更聪明，你更擅长找到这些简单的零件。

And then I think human institutions, human institutions are really difficult.

然后我认为人类机构，人类机构真的很困难。

It’s been a hard to get people.

很难找到人。

2:09:22 I won’t give specific examples, but it’s been hard to get people to adopt even the technologies that we’ve developed, even ones where the case for their efficacy is very, very strong.

我不会给出具体的例子，但即使是我们开发的技术，也很难让人们采用，即使是那些其功效非常非常强的技术。

People have concerns.

人们有顾虑。

They think things are conspiracy theories.

他们认为事情都是阴谋论。

It’s just been very difficult.

这非常困难。

It’s also been very difficult to get very simple things through the regulatory system.

通过监管系统获得非常简单的事情也非常困难。

And I don’t want to disparage anyone who works in regulatory systems of any technology.

我不想贬低任何在任何技术监管系统中工作的人。

There are hard they have to deal with.

他们必须面对困难。

They have to save lives.

他们必须拯救生命。

But the system as a whole, I think makes some obvious trade-offs that are very far from maximizing human welfare.

但我认为，整个系统做出了一些明显的权衡，而这些权衡与人类福利最大化相去甚远。

And so, if we bring AI systems into these human systems, often the level of intelligence may just not be the limiting factor.

因此，如果我们将人工智能系统引入这些人类系统，通常智能水平可能并不是限制因素。

It just may be that it takes a long time to do something.

可能只是做某事需要很长时间。

Now, if the AI system circumvented all governments, if it just said, “I’m dictator of the world and I’m going to do whatever,” some of these things it could do.

现在，如果人工智能系统绕过所有政府，如果它只是说，“我是世界的独裁者，我会做任何事情”，其中一些事情它可以做。

2:10:33 Again, the things have to do with complexity.

同样，事情与复杂性有关。

I still think a lot of things would take a while.

我仍然认为很多事情需要一段时间。

I don’t think it helps that the AI systems can produce a lot of energy or go to the moon.

我认为人工智能系统能够产生大量能源或登上月球并没有什么帮助。

Like some people in comments responded to the essay saying the AI system can produce a lot of energy and smarter AI systems.

就像评论中的一些人对这篇文章的回应说，人工智能系统可以产生大量的能源和更智能的人工智能系统。

That’s missing the point.

这没有抓住重点。

That kind of cycle doesn’t solve the key problems that I’m talking about here.

这种循环并不能解决我在这里谈论的关键问题。

So I think a bunch of people missed the point there.

所以我认为很多人都忽略了这一点。

But even if it were completely unaligned and could get around all these human obstacles it would have trouble.

但即使它完全不结盟并且能够绕过所有这些人类障碍，它也会遇到麻烦。

2:11:04 But again, if you want this to be an AI system that doesn’t take over the world, that doesn’t destroy humanity, then basically it’s going to need to follow basic human laws.

但同样，如果你希望这是一个不会接管世界、不会毁灭人类的人工智能系统，那么基本上它就需要遵循基本的人类法则。

If we want to have an actually good world, we’re going to have to have an AI system that interacts with humans, not one that creates its own legal system, or disregards all the laws or all of that.

如果我们想要拥有一个真正美好的世界，我们就必须拥有一个能够与人类互动的人工智能系统，而不是一个创建自己的法律体系或无视所有法律或所有这些的系统。

So as inefficient as these processes are, we’re going to have to deal with them, because there needs to be some popular and democratic legitimacy in how these systems are rolled out.

因此，尽管这些流程效率低下，我们还是不得不处理它们，因为这些系统的推出方式需要有一些受欢迎和民主的合法性。

We can’t have a small group of people who are developing these systems say, “This is what’s best for everyone.” I think it’s wrong, and I think in practice it’s not going to work anyway.

我们不能让一小群开发这些系统的人说：“这对每个人来说都是最好的。”我认为这是错误的，而且我认为实际上它是行不通的。

So you put all those things together and we’re not going change the world and upload everyone in five minutes.

所以你把所有这些东西放在一起，我们不会改变世界并在五分钟内上传所有内容。

A, I don’t think it’s going to happen and B, to the extent that it could happen.,It’s not the way to lead to a good world.

A，我认为这不会发生，B，就可能发生的程度而言。,这不是通向美好世界的方法。

So that’s on one side.

这是一方面。

2:12:07 On the other side, there’s another set of perspectives, which I have actually in some ways more sympathy for, which is, look, we’ve seen big productivity increases before.

另一方面，还有另一组观点，实际上在某些方面我更同情它们，那就是，看，我们以前已经看到生产力的大幅提高。

Economists are familiar with studying the productivity increases that came from the computer revolution and internet revolution.

经济学家熟悉计算机革命和互联网革命带来的生产率提高的研究。

And generally those productivity increases were underwhelming.

一般来说，生产率的提高并不令人印象深刻。

They were less than you might imagine.

他们比你想象的要少。

There was a quote from Robert Solow, “You see the computer revolution everywhere except the productivity statistics.” So why is this the case?

引用罗伯特·索洛的一句话：“除了生产力统计数据之外，你到处都可以看到计算机革命。”那么为什么会出现这样的情况呢？

People point to the structure of firms, the structure of enterprises, how slow it’s been to roll out our existing technology to very poor parts of the world, which I talk about in the essay.

人们指出公司的结构、企业的结构，以及将我们现有的技术推广到世界上非常贫穷的地区的速度有多慢，这些都是我在文章中谈到的。

How do we get these technologies to the poorest parts of the world that are behind on cell phone technology, computers, medicine, let alone newfangled AI that hasn’t been invented yet.

我们如何将这些技术带到世界上最贫穷的地区，这些地区在手机技术、计算机、医学方面落后，更不用说尚未发明的新型人工智能了。

2:13:04 So you could have a perspective that’s like, “Well, this is amazing technically, but it’s all or nothing burger.

所以你可能会有这样的观点：“嗯，这在技术上是令人惊叹的，但它要么全有要么全无汉堡。

I think Tyler Cowen who wrote something in response to my essay has that perspective.

我认为泰勒·考恩（Tyler Cowen）写了一些东西来回应我的文章，他也有这样的观点。

I think he thinks the radical change will happen eventually, but he thinks it’ll take 50 or 100 years.

我认为他认为彻底的改变最终会发生，但他认为这需要 50 或 100 年的时间。

And you could have even more static perspectives on the whole thing.

你甚至可以对整个事情有更静态的看法。

I think there’s some truth to it.

我认为这有一定道理。

I think the time scale is just too long and I can see it.

我认为时间尺度太长了，我可以看到它。

I can actually see both sides with today’s AI.

我实际上可以通过今天的人工智能看到两面。

So a lot of our customers are large enterprises who are used to doing things a certain way.

所以我们的很多客户都是大型企业，他们习惯于以某种方式做事。

I’ve also seen it in talking to governments, right?

我在与政府交谈时也看到过这一点，对吗？

Those are prototypical institutions, entities that are slow to change.

这些都是典型的机构，是变化缓慢的实体。

But, the dynamic I see over and over again is yes, it takes a long time to move the ship.

但是，我一遍又一遍地看到的动态是肯定的，移动船需要很长时间。

Yes.

是的。

There’s a lot of resistance and lack of understanding.

存在很多阻力和缺乏理解。

2:13:58 But, the thing that makes me feel that progress will in the end happen moderately fast, not incredibly fast, but moderately fast, is that you talk to …

但是，让我觉得进展最终会以中等速度发生的事情，不是令人难以置信的快，而是中等速度，是你与......

What I find is I find over and over again, again in large companies, even in governments which have been actually surprisingly forward leaning, you find two things that move things forward.

我发现，我一次又一次地发现，在大公司中，甚至在实际上出人意料地向前倾斜的政府中，你会发现有两件事可以推动事情向前发展。

One, you find a small fraction of people within a company, within a government, who really see the big picture, who see the whole scaling hypothesis, who understand where AI is going, or at least understand where it’s going within their industry.

第一，你会发现公司内部、政府内部的一小部分人真正看到了大局，看到了整个扩展假设，了解人工智能的发展方向，或者至少了解人工智能在其行业内的发展方向。

And there are a few people like that within the current US government who really see the whole picture.

在美国现任政府中，也有一些这样的人真正看到了全局。

And those people see that this is the most important thing in the world until they agitate for it.

这些人认为这是世界上最重要的事情，直到他们鼓动起来。

And the thing they alone are not enough to succeed, because there are a small set of people within a large organization.

而他们自己的事情还不足以成功，因为一个大组织内只有一小群人。

2:14:51 But, as the technology starts to roll out, as it succeeds in some places in the folks who are most willing to adopt it, the specter of competition gives them a wind at their backs, because they can point within their large organization.

但是，随着技术开始推广，当它在最愿意采用它的人们的某些地方取得成功时，竞争的幽灵给他们带来了后风，因为他们可以在他们的大型组织中指出。

They can say, “Look, these other guys are doing this.” One bank can say, “Look, this newfangled hedge fund is doing this thing.

他们可以说：“看，其他人也在这么做。”一家银行可以说：“看，这个新型对冲基金正在做这件事。

They’re going to eat our lunch.” In the US, we can say we’re afraid China’s going to get there before we are.

他们要吃我们的午餐。”在美国，我们可以说我们担心中国会先于我们实现这一目标。

And that combination, the specter of competition plus a few visionaries within these, the organizations that in many ways are sclerotic, you put those two things together and it actually makes something happen.

这种结合，竞争的幽灵加上其中的一些有远见的人，这些组织在很多方面都是僵化的，你把这两件事放在一起，它实际上会让一些事情发生。

It’s interesting.

这很有趣。

It’s a balanced fight between the two, because inertia is very powerful, but eventually over enough time, the innovative approach breaks through.

这是两者之间的平衡之战，因为惯性非常强大，但最终经过足够的时间，创新方法会取得突破。

2:15:48 And I’ve seen that happen.

我已经看到这种情况发生了。

I’ve seen the arc of that over and over again, and it’s like the barriers are there, the barriers to progress, the complexity, not knowing how to use the model, how to deploy them are there.

我一遍又一遍地看到这种弧线，就好像障碍在那里，进步的障碍，复杂性，不知道如何使用模型，如何部署它们。

And for a bit it seems like they’re going to last forever, change doesn’t happen.

有一段时间，它们似乎会永远持续下去，改变不会发生。

But, then eventually change happens and always comes from a few people.

但是，最终改变会发生，而且总是来自少数人。

I felt the same way when I was an advocate of the scaling hypothesis within the AI field itself and others didn’t get it.

当我是人工智能领域内缩放假设的倡导者而其他人不明白时，我也有同样的感觉。

It felt like no one would ever get it.

感觉好像没有人能得到它。

Then it felt like we had a secret almost no one ever had.

然后感觉就像我们拥有了一个几乎无人知晓的秘密。

And then, a couple years later, everyone has the secret.

几年后，每个人都知道了这个秘密。

And so, I think that’s how it’s going to go with deployment AI in the world.

因此，我认为这就是世界上人工智能部署的发展方向。

The barriers are going to fall apart gradually and then all at once.

障碍将逐渐瓦解，然后突然瓦解。

2:16:35 And so, I think this is going to be more, and this is just an instinct.

所以，我认为这会更多，这只是一种本能。

I could easily see how I’m wrong.

我很容易看出我错了。

I think it’s going to be more five or 10 years, as I say in the essay than it’s going to be 50 or 100 years.

正如我在文章中所说，我认为这将是五年或十年，而不是五十年或一百年。

I also think it’s going to be five or 10 years more than it’s going to be five or 10 hours, because I’ve just seen how human systems work.

我还认为这将比 5 或 10 小时多花 5 到 10 年，因为我刚刚看到了人类系统是如何工作的。

And I think a lot of these people who write down these differential equations, who say AI is going to make more powerful AI, who can’t understand how it could possibly be the case that these things won’t change so fast.

我认为很多写下这些微分方程的人，他们说人工智能将创造更强大的人工智能，他们无法理解这些事情怎么可能不会改变得这么快。

I think they don’t understand these things.

我认为他们不明白这些事情。

So what to you is the timeline to where we achieve AGI, A.K.A.

那么，对您来说，我们实现 AGI（A.K.A）的时间表是什么？

2:17:11 powerful AI, A.K.A.

强大的人工智能，A.K.A.

super useful AI?

超级有用的人工智能？

I’m going to start calling it that.

我将开始这样称呼它。

It’s a debate about naming.

这是一场关于命名的争论。

2:17:24 On pure intelligence smarter than a Nobel Prize winner in every relevant discipline and all the things we’ve said.

在纯粹的智力上，在每个相关学科和我们所说的所有事情上都比诺贝尔奖获得者聪明。

Modality, can go and do stuff on its own for days, weeks, and do biology experiments on its own in one …

模态，可以连续数天、数周独自做事情，并单独进行生物学实验……

You know what?

你知道吗？

Let’s just stick to biology, because you sold me on the whole biology and health section.

我们就专注于生物学吧，因为你把整个生物学和健康部分卖给了我。

And that’s so exciting from just …

这太令人兴奋了……

I was getting giddy from a scientific perspective.

从科学的角度来看，我感到头晕。

It made me want to be a biologist.

这让我想成为一名生物学家。

So no,.

所以不，。

2:17:56 No.

不。

This was the feeling I had when I was writing it, that it’s like, this would be such a beautiful future if we can just make it happen.

这就是我写这篇文章时的感觉，如果我们能够实现的话，这将是一个多么美好的未来。

If we can just get the landmines out of the way and make it happen.

如果我们能清除地雷并实现这一目标就好了。

There’s so much beauty and elegance and moral force behind it if we can just …

如果我们能够……，它的背后蕴含着如此多的美丽、优雅和道德力量。

And it’s something we should all be able to agree on.

这是我们都应该达成一致的事情。

As much as we fight about all these political questions, is this something that could actually bring us together?

尽管我们为所有这些政治问题争论不休，但这真的能让我们走到一起吗？

But you were asking when will we get this?

但你问我们什么时候能得到这个？

When?

什么时候？

2:18:32 When do you think?

你什么时候想？

Just putting numbers on the table.

只是把数字放在桌子上。

This is, of course, the thing I’ve been grappling with for many years, and I’m not at all confident.

当然，这是我多年来一直在努力解决的问题，而且我一点也不自信。

2:18:36 If I say 2026 or 2027, there will be a zillion people on Twitter who will be like, “AI CEO said 2026, 2020 …

如果我说 2026 年或 2027 年，Twitter 上会有无数人说，“AI 首席执行官说 2026 年、2020 年……

” and it’ll be repeated for the next two years that this is definitely when I think it’s going to happen.

”并且在接下来的两年里，我认为这肯定会发生，这一点将会重复。

So whoever’s exerting these clips will crop out the thing I just said and only say the thing I’m about to say.

因此，无论谁播放这些剪辑，都会剪掉我刚才说的内容，只说我要说的内容。

But I’ll just say it anyway-

但我还是要说一下——

Have fun with it.

玩得开心。

So if you extrapolate the curves that we’ve had so far.

因此，如果你推断出我们迄今为止的曲线。

2:19:08 Right?

对吗？

If you say, “Well, I don’t know.

如果你说：“好吧，我不知道。

We’re starting to get to PhD level, and last year we were at undergraduate level and the year before we were at the level of a high school student.” Again, you can quibble with at what tasks and for what we’re still missing modalities, but those are being added.

我们开始进入博士学位阶段，去年我们是本科生水平，前年是高中生水平。”同样，您可能会质疑我们仍然缺少哪些任务和哪些模式，但这些正在被添加。

Computer use was added, like ImageEn was added, image generation has been added.

添加了计算机使用，就像添加了 ImageEn、添加了图像生成一样。

And this is totally unscientific, but if you just eyeball the rate at which these capabilities are increasing, it does make you think that we’ll get there by 2026 or 2027.

这是完全不科学的，但如果你只关注这些能力增长的速度，它确实会让你认为我们将在 2026 年或 2027 年实现这一目标。

Again, lots of things could derail it.

同样，很多事情都可能使其脱轨。

We could run out of data.

我们可能会耗尽数据。

We might not be able to scale clusters as much as we want.

我们可能无法按照我们想要的方式扩展集群。

Maybe Taiwan gets blown up or something, and then we can’t produce as many GPUs as we want.

也许台湾被炸毁了或者其他什么原因，然后我们就无法生产我们想要的那么多GPU。

So there are all-

所以就有了——

2:20:00 Then we can’t produce as many GPUs as we want.

那么我们就无法生产出我们想要的那么多 GPU。

So there are all kinds of things that could derail the whole process.

因此，有各种各样的事情可能会破坏整个过程。

So I don’t fully believe the straight line extrapolation, but if you believe the straight line extrapolation, we’ll get there in 2026 or 2027.

所以我并不完全相信直线外推法，但如果你相信直线外推法，我们将在 2026 年或 2027 年实现这一目标。

I think the most likely is that there are some mild delay relative to that.

我认为最有可能的是相对于此存在一些轻微的延迟。

I don’t know what that delay is, but I think it could happen on schedule.

我不知道延迟是什么，但我认为它可能会按计划发生。

I think there could be a mild delay.

我认为可能会有轻微的延迟。

I think there are still worlds where it doesn’t happen in a hundred years.

我认为仍然有一些世界一百年后不会发生这种情况。

The number of those worlds is rapidly decreasing.

这些世界的数量正在迅速减少。

We are rapidly running out of truly convincing blockers, truly compelling reasons why this will not happen in the next few years.

我们正在迅速耗尽真正令人信服的阻碍因素，以及在未来几年内不会发生这种情况的真正令人信服的理由。

2:20:39 There were a lot more in 2020, although my guess, my hunch at that time was that we’ll make it through all those blockers.

2020 年还会有更多障碍，尽管我当时的猜测和预感是我们将克服所有这些障碍。

So sitting as someone who has seen most of the blockers cleared out of the way, I suspect, my hunch, my suspicion is that the rest of them will not block us.

因此，作为一个已经看到大部分阻碍者被清除的人，我怀疑，我的预感，我的怀疑是，其余的人不会阻碍我们。

But look, at the end of the day, I don’t want to represent this as a scientific prediction.

但你看，归根结底，我不想将其视为科学预测。

People call them scaling laws.

人们称它们为缩放定律。

That’s a misnomer.

这是用词不当。

Like Moore’s law is a misnomer.

就像摩尔定律是一个用词不当一样。

Moore’s laws, scaling laws, they’re not laws of the universe.

摩尔定律、缩放定律，它们不是宇宙定律。

They’re empirical regularities.

它们是经验规律。

I am going to bet in favor of them continuing, but I’m not certain of that.

我打赌他们会继续下去，但我对此并不确定。

So you extensively described sort of the compressed 21st century, how AGI will help set forth a chain of breakthroughs in biology and medicine that help us in all these kinds of ways that I mentioned.

因此，您广泛描述了压缩的 21 世纪，AGI 将如何帮助在生物学和医学领域实现一系列突破，以我提到的所有这些方式帮助我们。

2:21:15 What are the early steps it might do?

它可能采取哪些早期步骤？

And by the way, I asked Claude good questions to ask you and Claude told me to ask, what do you think is a typical day for a biologist working on AGI look like in this future?

顺便说一句，我问了克劳德一些很好的问题，克劳德让我问，你认为未来从事 AGI 工作的生物学家典型的一天会是什么样子？

Yeah, yeah.

是啊，是啊。

Claude is curious.

克劳德很好奇。

Well, let me start with your first questions and then I’ll answer that.

好吧，让我从你的第一个问题开始，然后我会回答。

2:21:48 Claude wants to know what’s in his future, right?

克劳德想知道他的未来，对吗？

Exactly.

正是如此。

Who am I going to be working with?

我要和谁一起工作？

Exactly.

正是如此。

So I think one of the things when I went hard on in the essay is let me go back to this idea of, because it’s really had an impact on me, this idea that within large organizations and systems, there end up being a few people or a few new ideas who cause things to go in a different direction than they would’ve before who kind of disproportionately affect the trajectory.

所以我认为，当我在文章中继续努力时，让我回到这个想法，因为它确实对我产生了影响，这个想法是，在大型组织和系统中，最终会有一些人或一些新想法导致事情朝着与以前不同的方向发展，他们对轨迹产生了不成比例的影响。

2:21:56 There’s a bunch of the same thing going on, right?

有很多相同的事情正在发生，对吗？

If you think about the health world, there’s like trillions of dollars to pay out Medicare and other health insurance and then the NIH is 100 billion.

如果你考虑一下健康领域，就会发现有数万亿美元用于支付医疗保险和其他健康保险，而 NIH 则有 1000 亿美元。

And then if I think of the few things that have really revolutionized anything, it could be encapsulated in a small fraction of that.

然后，如果我想到真正彻底改变任何事物的几件事，它可以被封装在其中的一小部分中。

And so when I think of where will AI have an impact, I’m like, “Can AI turn that small fraction into a much larger fraction and raise its quality?”

因此，当我想到人工智能将在哪些方面产生影响时，我想，“人工智能能否将这一小部分变成更大的部分并提高其质量？”

2:22:49 And within biology, my experience within biology is that the biggest problem of biology is that you can’t see what’s going on.

在生物学领域，我在生物学领域的经验是，生物学最大的问题是你看不到正在发生的事情。

You have very little ability to see what’s going on and even less ability to change it, right?

你几乎没有能力看到正在发生的事情，更没有能力改变它，对吗？

What you have is this.

你拥有的就是这个。

From this, you have to infer that there’s a bunch of cells that within each cell is 3 billion base pairs of DNA built according to a genetic code.

由此，你必须推断出有一堆细胞，每个细胞内都有根据遗传密码构建的 30 亿个碱基对的 DNA。

And there are all these processes that are just going on without any ability of us on unaugmented humans to affect it.

所有这些过程都在发生，而我们对未增强的人类没有任何能力去影响它。

These cells are dividing.

这些细胞正在分裂。

Most of the time that’s healthy, but sometimes that process goes wrong and that’s cancer.

大多数时候这是健康的，但有时这个过程出了问题，那就是癌症。

The cells are aging, your skin may change color, develops wrinkles as you age, and all of this is determined by these processes.

随着年龄的增长，细胞会老化，你的皮肤可能会变色、出现皱纹，所有这一切都是由这些过程决定的。

All these proteins being produced, transported to various parts of the cells binding to each other.

所有这些蛋白质都被产生、运输到细胞的各个部分并相互结合。

2:23:50 And in our initial state about biology, we didn’t even know that these cells existed.

在我们对生物学的最初认识中，我们甚至不知道这些细胞的存在。

We had to invent microscopes to observe the cells.

我们必须发明显微镜来观察细胞。

We had to invent more powerful microscopes to see below the level of the cell to the level of molecules.

我们必须发明更强大的显微镜来观察细胞水平以下到分子水平。

We had to invent X-ray crystallography to see the DNA.

我们必须发明 X 射线晶体学来观察 DNA。

We had to invent gene sequencing to read the DNA.

我们必须发明基因测序来读取 DNA。

Now we had to invent protein folding technology to predict how it would fold and how these things bind to each other.

现在我们必须发明蛋白质折叠技术来预测它如何折叠以及这些东西如何彼此结合。

We had to invent various techniques for now we can edit the DNA as of with CRISPR as of the last 12 years.

我们必须发明各种技术，才能在过去 12 年中使用 CRISPR 编辑 DNA。

So the whole history of biology, a whole big part of the history is basically our ability to read and understand what’s going on and our ability to reach in and selectively change things.

因此，整个生物学的历史，历史的很大一部分，基本上就是我们阅读和理解正在发生的事情的能力，以及我们进入并有选择地改变事物的能力。

And my view is that there’s so much more we can still do there.

我的观点是，我们仍然可以做更多的事情。

2:24:48 You can do CRISPR, but you can do it for your whole body.

你可以做 CRISPR，但你可以为你的整个身体做这件事。

Let’s say I want to do it for one particular type of cell and I want the rate of targeting the wrong cell to be very low.

假设我想针对一种特定类型的细胞进行此操作，并且我希望针对错误细胞的比率非常低。

That’s still a challenge.

这仍然是一个挑战。

That’s still things people are working on.

这仍然是人们正在努力的事情。

That’s what we might need for gene therapy for certain diseases.

这就是我们对某些疾病进行基因治疗可能需要的东西。

The reason I’m saying all of this, it goes beyond this to gene sequencing, to new types of nanomaterials for observing what’s going on inside cells, for antibody drug conjugates.

我之所以说这一切，是因为它不仅仅涉及基因测序、用于观察细胞内部发生的情况的新型纳米材料以及抗体药物偶联物。

The reason I’m saying all this is that this could be a leverage point for the AI systems, right?

我之所以这么说是因为这可能是人工智能系统的一个杠杆点，对吗？

That the number of such inventions, it’s in the mid double digits or something, mid double digits, maybe low triple digits over the history of biology.

在生物学史上，此类发明的数量大约是中两位数，甚至可能是低三位数。

Let’s say I have a million of these AIs like can they discover a thousand working together or can they discover thousands of these very quickly and does that provide a huge lever?

假设我有一百万个这样的人工智能，他们能否发现一千个一起工作，或者他们能否很快发现数千个这样的人工智能，这是否提供了巨大的杠杆？

2:25:45 Instead of trying to leverage two trillion a year we spend on Medicare or whatever, can we leverage the 1 billion a year that’s spent to discover but with much higher quality?

与其试图利用我们每年在医疗保险或其他方面花费的 2 万亿美元，不如利用每年用于发现的 10 亿美元，但质量要高得多？

And so what is it like being a scientist that works with an AI system?

那么，作为一名使用人工智能系统的科学家是什么感觉呢？

The way I think about it actually is, well, so I think in the early stages, the AIs are going to be like grad students.

我实际上的想法是，嗯，所以我认为在早期阶段，人工智能将像研究生一样。

You’re going to give them a project.

你要给他们一个项目。

You’re going to say, “I’m the experienced biologist.

你会说：“我是经验丰富的生物学家。

I’ve set up the lab.” The biology professor or even the grad students themselves will say, “Here’s what you can do with an AI…

我已经建立了实验室。”生物学教授甚至研究生自己都会说：“这就是你可以用人工智能做的事情......

AI system, I’d like to study this.” And the AI system, it has all the tools.

AI系统，我想研究这个。”还有人工智能系统，它拥有所有工具。

It can look up all the literature to decide what to do.

它可以查找所有文献来决定做什么。

It can look at all the equipment.

它可以查看所有设备。

It can go to a website and say, “Hey, I’m going to go to Thermo Fisher or whatever the dominant lab equipment company is today.

它可以访问一个网站并说：“嘿，我要去赛默飞世尔或当今占主导地位的实验室设备公司。

My time was Thermo Fisher.

我的时代是赛默飞世尔。

2:26:48 I’m going to order this new equipment to do this.

我将订购新设备来完成这项工作。

I’m going to run my experiments.

我要进行我的实验。

I’m going to write up a report about my experiments.

我要写一份关于我的实验的报告。

I’m going to inspect the images for contamination.

我要检查图像是否受到污染。

I’m going to decide what the next experiment is.

我将决定下一个实验是什么。

I’m going to write some code and run a statistical analysis.

我将编写一些代码并运行统计分析。

All the things a grad student would do that’ll be a computer with an AI that the professor talks to every once in a while and it says, “This is what you’re going to do today.” The AI system comes to it with questions.

研究生要做的所有事情都将是一台带有人工智能的计算机，教授每隔一段时间就会与它交谈，它会说：“这就是你今天要做的事情。”人工智能系统会提出问题。

When it’s necessary to run the lab equipment, it may be limited in some ways.

当需要运行实验室设备时，可能会受到某些方面的限制。

It may have to hire a human lab assistant to do the experiment and explain how to do it or it could use advances in lab automation that are gradually being developed or have been developed over the last decade or so and will continue to be developed.

它可能需要聘请一名人类实验室助理来进行实验并解释如何进行实验，或者它可以利用实验室自动化方面的进步，这些进步正在逐渐开发或在过去十年左右已经开发并将继续开发。

2:27:38 And so it’ll look like there’s a human professor and 1,000 AI grad students and if you go to one of these Nobel Prize winning biologists or so, you’ll say, “Okay, well, you had like 50 grad students.

因此，看起来就像有一位人类教授和 1,000 名人工智能研究生，如果你去找一位诺贝尔奖获得者左右的生物学家，你会说，“好吧，好吧，你有大约 50 名研究生。

Well, now you have 1,000 and they’re smarter than you are by the way.” Then I think at some point it’ll flip around where the AI systems will be the PIs, will be the leaders, and they’ll be ordering humans or other AI systems around.

好吧，现在你有 1000 个，顺便说一句，他们比你聪明。”然后我认为在某个时候，情况将会发生逆转，人工智能系统将成为 PI，成为领导者，他们将指挥人类或其他人工智能系统。

So I think that’s how it’ll work on the research side.

所以我认为这就是研究方面的运作方式。

And there would be the inventors of a CRISPR type technology.

还会有 CRISPR 类型技术的发明者。

They would be the inventors of a CRISPR type technology.

他们将成为 CRISPR 类型技术的发明者。

2:28:08 And then I think, as I say in the essay, we’ll want to turn, probably turning loose is the wrong term, but we’ll want to harness the AI systems to improve the clinical trial system as well.

然后我认为，正如我在文章中所说，我们想要转变，可能“放松”这个词是错误的，但我们也想要利用人工智能系统来改进临床试验系统。

There’s some amount of this that’s regulatory, that’s a matter of societal decisions and that’ll be harder.

其中有一些是监管方面的，这是社会决策的问题，这会更困难。

But can we get better at predicting the results of clinical trials?

但我们能更好地预测临床试验的结果吗？

Can we get better at statistical design so that clinical trials that used to require 5,000 people and therefore needed $100 million in a year to enroll them, now they need 500 people in two months to enroll them?

我们能否在统计设计方面做得更好，以便临床试验过去需要 5,000 人，因此一年内需要 1 亿美元来招募他们，现在他们需要两个月内招募 500 人？

That’s where we should start.

这就是我们应该开始的地方。

And can we increase the success rate of clinical trials by doing things in animal trials that we used to do in clinical trials and doing things in simulations that we used to do in animal trials?

我们是否可以通过在动物试验中做我们过去在临床试验中做的事情以及在模拟中做我们过去在动物试验中做的事情来提高临床试验的成功率？

Again, we won’t be able to simulate at all.

同样，我们根本无法模拟。

AI is not God, but can we shift the curve substantially and radically?

人工智能不是上帝，但我们能否从根本上大幅改变这一趋势？

So I don’t know, that would be my picture.

所以我不知道，那是我的照片。

Doing in vitro and doing it.

在体外进行并进行。

2:29:15 I mean you’re still slowed down.

我的意思是你仍然放慢了速度。

It still takes time, but you can do it much, much faster.

这仍然需要时间，但你可以做得更快。

Yeah, yeah.

是啊是啊。

2:29:21 Can we just one step at a time and can that add up to a lot of steps?

我们可以一次只迈出一步吗？这样加起来可以有很多步骤吗？

Even though though we still need clinical trials, even though we still need laws, even though the FDA and other organizations will still not be perfect, can we just move everything in a positive direction and when you add up all those positive directions, do you get everything that was going to happen from here to 2100 instead happens from 2027 to 2032 or something?

尽管我们仍然需要临床试验，尽管我们仍然需要法律，尽管 FDA 和其他组织仍然不完美，但我们能否让一切朝着积极的方向发展？当你把所有这些积极的方向加起来时，你是否会得到从现在到 2100 年将要发生的一切，而不是从 2027 年到 2032 年发生的事情？

Another way that I think the world might be changing with AI even today, but moving towards this future of the powerful super useful AI is programming.

我认为即使在今天，世界也可能会随着人工智能而改变，但走向强大、超级有用的人工智能的未来是编程。

2:29:46 So how do you see the nature of programming because it’s so intimate to the actual act of building AI.

那么你如何看待编程的本质，因为它与构建人工智能的实际行为如此密切。

How do you see that changing for us humans?

您如何看待我们人类的这种变化？

I think that’s going to be one of the areas that changes fastest for two reasons.

我认为这将成为变化最快的领域之一，原因有两个。

2:30:09 One, programming is a skill that’s very close to the actual building of the AI.

第一，编程是一种与人工智能的实际构建非常接近的技能。

So the farther a skill is from the people who are building the AI, the longer it’s going to take to get disrupted by the AI.

因此，一项技能距离构建人工智能的人越远，被人工智能颠覆所需的时间就越长。

I truly believe that AI will disrupt agriculture.

我坚信人工智能将颠覆农业。

Maybe it already has in some ways, but that’s just very distant from the folks who are building AI, and so I think it’s going to take longer.

也许在某些方面已经做到了，但这距离构建人工智能的人们还很遥远，所以我认为这需要更长的时间。

But programming is the bread and butter of a large fraction of the employees who work at Anthropic and at the other companies, and so it’s going to happen fast.

但编程是 Anthropic 和其他公司大部分员工的谋生手段，因此它会很快发生。

The other reason it’s going to happen fast is with programming, you close the loop both when you’re training the model and when you’re applying the model.

它会快速发生的另一个原因是通过编程，在训练模型和应用模型时都可以关闭循环。

2:30:52 The idea that the model can write the code means that the model can then run the code and then see the results and interpret it back.

模型可以编写代码的想法意味着模型可以运行代码，然后查看结果并将其解释回来。

And so it really has an ability unlike hardware, unlike biology, which we just discussed, the model has an ability to close the loop.

因此，它确实具有与硬件不同的能力，与我们刚才讨论的生物学不同，该模型具有闭环的能力。

And so I think those two things are going to lead to the model getting good at programming very fast.

所以我认为这两件事将导致模型非常快速地擅长编程。

As I saw on typical real-world programming tasks, models have gone from 3% in January of this year to 50% in October of this year.

正如我在典型的现实世界编程任务中看到的那样，模型已从今年 1 月的 3% 上升到今年 10 月的 50%。

So we’re on that S-curve where it’s going to start slowing down soon because you can only get to 100%.

所以我们正处于 S 曲线上，很快就会开始减速，因为你只能达到 100%。

But I would guess that in another 10 months, we’ll probably get pretty close.

但我猜想再过 10 个月，我们可能会非常接近这一目标。

We’ll be at least 90%.

我们至少会达到90%。

So again, I would guess, I don’t know how long it’ll take, but I would guess again, 2026, 2027 Twitter people who crop out these numbers and get rid of the caveats, I don’t know.

所以，我再次猜测，我不知道需要多长时间，但我再次猜测，2026 年、2027 年 Twitter 上的人会剪掉这些数字并消除警告，我不知道。

2:31:53 I don’t like you, go away.

我不喜欢你，走开。

I would guess that the kind of task that the vast majority of coders do, AI can probably, if we make the task very narrow, just write code, AI systems will be able to do that.

我猜想，绝大多数编码员所做的那种任务，人工智能可能可以，如果我们把任务变得非常狭窄，只需编写代码，人工智能系统就能够做到这一点。

Now that said, I think comparative advantage is powerful.

话虽如此，我认为比较优势是强大的。

We’ll find that when AIs can do 80% of a coder’s job, including most of it that’s literally write code with a given spec, we’ll find that the remaining parts of the job become more leveraged for humans, right?

我们会发现，当人工智能可以完成 80% 的编码员工作（包括大部分按照给定规范编写代码）时，我们会发现工作的其余部分对人类来说变得更加有用，对吗？

Humans, there’ll be more about high level system design or looking at the app and is it architected well and the design and UX aspects and eventually AI will be able to do those as well.

人类将更多地关注高级系统设计或查看应用程序，它的架构是否良好，设计和用户体验方面以及最终人工智能也将能够做到这些。

That’s my vision of the powerful AI system.

这就是我对强大的人工智能系统的愿景。

But I think for much longer than we might expect, we will see that small parts of the job that humans still do will expand to fill their entire job in order for the overall productivity to go up.

但我认为，在比我们预期更长的时间内，我们将看到人类仍在从事的工作中的一小部分将扩大到填补他们的整个工作，从而提高整体生产力。

That’s something we’ve seen.

这是我们所看到的。

It used to be that writing and editing letters was very difficult and writing the print was difficult.

过去，写信和编辑信件非常困难，写印刷品也很困难。

Well, as soon as you had word processors and then computers and it became easy to produce work and easy to share it, then that became instant and all the focus was on the ideas.

好吧，一旦有了文字处理器，然后是计算机，创作工作和分享变得容易，那么一切就变得即时，所有的焦点都集中在想法上。

So this logic of comparative advantage that expands tiny parts of the tasks to large parts of the tasks and creates new tasks in order to expand productivity, I think that’s going to be the case.

因此，这种将任务的微小部分扩展到任务的大部分并创建新任务以提高生产率的比较优势逻辑，我认为情况将会如此。

2:33:32 Again, someday AI will be better at everything and that logic won’t apply, and then humanity will have to think about how to collectively deal with that and we’re thinking about that every day and that’s another one of the grand problems to deal with aside from misuse and autonomy and we should take it very seriously.

再说一遍，有一天人工智能会在所有方面都变得更好，而这种逻辑将不再适用，然后人类将不得不考虑如何集体处理这个问题，我们每天都在思考这一点，这是除了滥用和自主之外需要处理的另一个重大问题，我们应该非常认真地对待它。

But I think in the near term, and maybe even in the medium term, medium term like 2, 3, 4 years, I expect that humans will continue to have a huge role and the nature of programming will change, but programming as a role, programming as a job will not change.

但我认为在短期内，甚至可能在中期，像2、3、4年这样的中期，我预计人类将继续发挥巨大的作用，编程的本质将会改变，但编程作为一种角色、编程作为一种工作不会改变。

It’ll just be less writing things line by line and it’ll be more macroscopic.

只是会减少逐行书写的次数，并且会更加宏观。

And I wonder what the future of IDEs looks like.

我想知道 IDE 的未来会是什么样子。

2:34:10 So the tooling of interacting with AI systems, this is true for programming and also probably true for in other contexts like computer use, but maybe domain specific, like we mentioned biology, it probably needs its own tooling about how to be effective.

因此，与人工智能系统交互的工具，对于编程来说是正确的，对于计算机使用等其他环境也可能如此，但也许特定领域，就像我们提到的生物学一样，它可能需要自己的工具来提高效率。

And then programming needs its own tooling.

然后编程需要自己的工具。

Is Anthropic going to play in that space of also tooling potentially?

Anthropic 是否会在工具领域发挥潜在作用？

I’m absolutely convinced that powerful IDEs, that there’s so much low-hanging fruit to be grabbed there that right now it’s just like you talk to the model and it talks back.

我绝对相信强大的 IDE，有很多唾手可得的成果，现在就像你与模型交谈，它会做出回应。

2:34:30 But look, I mean IDEs are great at lots of static analysis of so much is possible with static analysis like many bugs you can find without even writing the code.

但是看，我的意思是 IDE 非常擅长进行大量静态分析，通过静态分析可以实现很多功能，例如您甚至无需编写代码就可以找到许多错误。

Then IDEs are good for running particular things, organizing your code, measuring coverage of unit tests.

那么 IDE 非常适合运行特定的东西、组织代码、测量单元测试的覆盖率。

There’s so much that’s been possible with a normal IDEs.

普通 IDE 可以做很多事情。

Now you add something like, well, the model can now write code and run code.

现在您添加一些内容，例如模型现在可以编写代码并运行代码。

I am absolutely convinced that over the next year or two, even if the quality of the models didn’t improve, that there would be enormous opportunity to enhance people’s productivity by catching a bunch of mistakes, doing a bunch of grunt work for people, and that we haven’t even scratched the surface.

我绝对相信，在接下来的一两年里，即使模型的质量没有提高，也会有巨大的机会通过发现一堆错误、为人们做一堆繁重的工作来提高人们的生产力，而我们甚至还没有触及表面。

2:35:33 Anthropic itself, I mean you can’t say no…

人择本身，我的意思是你不能拒绝……

It’s hard to say what will happen in the future.

很难说未来会发生什么。

Currently, we’re not trying to make such IDEs ourself, rather we’re powering the companies like Cursor or Kognition or some of the other expo in the security space, others that I could mention as well that are building such things themselves on top of our API and our view has been let 1,000 flowers bloom.

目前，我们并没有尝试自己制作这样的 IDE，而是为 Cursor 或 Kognition 等公司或安全领域的其他一些博览会提供支持，我也可以提到的其他公司正在我们的 API 之上构建此类东西，我们的观点是让 1,000 朵花绽放。

We don’t internally have the resources to try all these different things.

我们内部没有资源来尝试所有这些不同的事情。

Let’s let our customers try it and we will see who succeeds and maybe different customers will succeed in different ways.

让我们的客户尝试一下，我们就会看到谁能成功，也许不同的客户会以不同的方式取得成功。

So I both think this is super promising and Anthropic isn’t eager to, at least right now, compete with all our companies in this space and maybe never.

所以我都认为这是非常有前途的，而且 Anthropic 并不急于（至少现在）与我们在这个领域的所有公司竞争，也许永远不会。

Yeah, it’s been interesting to watch Cursor try to integrate cloud successfully because it’s actually fascinating how many places it can help the programming experience.

是的，观看 Cursor 尝试成功集成云是很有趣的，因为它在很多地方可以帮助编程体验，这实际上很有趣。

2:36:27 It’s not as trivial.

这并不是那么微不足道的事情。

It is really astounding.

实在是令人惊叹不已。

2:36:37 I feel like as a CEO, I don’t get to program that much, and I feel like if six months from now I go back, it’ll be completely unrecognizable to me.

我觉得作为一名首席执行官，我不会进行太多编程，而且我觉得如果六个月后我回去，我会完全认不出来。

Exactly.

确切地。

2:36:45 In this world with super powerful AI that’s increasingly automated, what’s the source of meaning for us humans?

在这个人工智能日益自动化的世界里，对于我们人类来说，意义的来源是什么？

Work is a source of deep meaning for many of us.

对我们许多人来说，工作是深刻意义的源泉。

Where do we find the meaning?

我们到哪里去寻找意义呢？

This is something that I’ve written about a little bit in the essay, although I actually give it a bit short shrift, not for any principled reason, but this essay, if you believe it was originally going to be two or three pages, I was going to talk about it at all hands.

这是我在文章中写过的一些内容，尽管我实际上对它有点不屑一顾，不是出于任何原则性原因，但是这篇文章，如果你认为它最初只有两到三页，我会全力谈论它。

2:37:01 And the reason I realized it was an important underexplored topic is that I just kept writing things and I was just like, “Oh man, I can’t do this justice.” And so the thing ballooned to 40 or 50 pages and then when I got to the work and meaning section, I’m like, “Oh man, this isn’t going to be 100 pages.” I’m going to have to write a whole other essay about that.

我之所以意识到这是一个尚未得到充分探索的重要话题，是因为我一直在写东西，然后我就想，“天哪，我不能公正地对待这个问题。”所以这个东西膨胀到了 40 或 50 页，然后当我到达工作和意义部分时，我想，“天哪，这不会是 100 页。”我将不得不写一篇关于此的另一篇文章。

But meaning is actually interesting because you think about the life that someone lives or something, or let’s say you were to put me in, I don’t know, like a simulated environment or something where I have a job and I’m trying to accomplish things and I don’t know, I do that for 60 years and then you’re like, “Oh, oops, this was actually all a game,” right?

但意义实际上很有趣，因为你想到某人的生活或某事，或者假设你要把我放进去，我不知道，就像一个模拟环境，或者我有一份工作，我正在努力完成一些事情，但我不知道，我这样做了 60 年，然后你会想，“哦，哎呀，这实际上只是一场游戏，”对吗？

2:37:56 Does that really kind of rob you of the meaning of the whole thing?

这真的让你失去了整件事的意义吗？

I still made important choices, including moral choices.

我仍然做出了重要的选择，包括道德选择。

I still sacrificed.

我还是牺牲了。

I still had to gain all these skills or just a similar exercise.

我仍然需要获得所有这些技能或只是进行类似的练习。

Think back to one of the historical figures who discovered electromagnetism or relativity or something.

回想一下发现电磁学或相对论或其他东西的历史人物之一。

If you told them, “Well, actually 20,000 years ago, some alien on this planet discovered this before you did,” does that rob the meaning of the discovery?

如果你告诉他们，“嗯，实际上 20,000 年前，这个星球上的某个外星人在你之前发现了这个”，这是否剥夺了这一发现的意义？

It doesn’t really seem like it to me, right?

对我来说，这确实不像，对吧？

It seems like the process is what matters and how it shows who you are as a person along the way and how you relate to other people and the decisions that you make along the way.

似乎过程才是最重要的，它如何展示你作为一个人的身份，以及你与他人的关系以及你在此过程中做出的决定。

Those are consequential.

这些都是必然的。

I could imagine if we handle things badly in an AI world, we could set things up where people don’t have any long-term source of meaning or any, but that’s more a set of choices we make that’s more a set of the architecture of society with these powerful models.

我可以想象，如果我们在人工智能世界中处理不好，我们可能会在人们没有任何长期意义来源的地方设置一些东西，但这更多的是我们做出的一组选择，更多的是一组具有这些强大模型的社会架构。

If we design it badly and for shallow things, then that might happen.

如果我们设计得很糟糕并且只做肤浅的事情，那么这种情况就可能发生。

I would also say that most people’s lives today, while admirably, they work very hard to find meaning in those lives.

我还想说，大多数人今天的生活令人钦佩的是，他们非常努力地寻找生活的意义。

Like look, we who are privileged and who are developed these technologies, we should have empathy for people not just here, but in the rest of the world who spend a lot of their time scraping by to survive, assuming we can distribute the benefits of this technology to everywhere, their lives are going to get a hell of a lot better and meaning will be important to them as it is important to them now.

就像我们这些享有特权并开发了这些技术的人一样，我们不仅应该同情这里的人们，而且应该同情世界其他地方的人们，他们花了很多时间勉强生存，假设我们可以将这项技术的好处分配给世界各地，他们的生活将会变得更好，意义对他们来说很重要，就像现在对他们很重要一样。

2:39:41 But we should not forget the importance of that and that the idea of meaning as the only important thing is in some ways an artifact of a small subset of people who have been economically fortunate.

但我们不应该忘记这一点的重要性，并且“意义作为唯一重要的事情”的观念在某种程度上是一小部分经济上富裕的人的产物。

But I think all of that said, I think a world is possible with powerful AI that not only has as much meaning for everyone, but that has more meaning for everyone that can allow everyone to see worlds and experiences that it was either possible for no one to see or a possible for very few people to experience.

但我认为所有这些都说明了，我认为一个拥有强大人工智能的世界是可能的，它不仅对每个人都有同样的意义，而且对每个人都有更多的意义，可以让每个人看到没有人可能看到的世界和体验，或者很少有人可以体验到的世界和体验。

2:40:21 So I am optimistic about meaning.

所以我对意义持乐观态度。

I worry about economics and the concentration of power.

我担心经济和权力集中。

That’s actually what I worry about more.

其实这才是我更担心的。

I worry about how do we make sure that that fair world reaches everyone.

我担心我们如何确保每个人都能享受到这个公平的世界。

When things have gone wrong for humans, they’ve often gone wrong because humans mistreat other humans.

当人类出了问题时，往往是因为人类虐待了其他人。

That is maybe in some ways even more than the autonomous risk of AI or the question of meaning.

在某些方面，这可能比人工智能的自主风险或意义问题还要严重。

That is the thing I worry about most, the concentration of power, the abuse of power, structures like autocracies and dictatorships where a small number of people exploits a large number of people.

这是我最担心的事情，权力的集中，权力的滥用，像独裁和独裁这样的结构，少数人剥削大量人。

I’m very worried about that.

我对此非常担心。

And AI increases the amount of power in the world, and if you concentrate that power and abuse that power, it can do immeasurable damage.

人工智能增加了世界上的力量，如果你集中这种力量并滥用这种力量，它会造成不可估量的损害。

Yes, it’s very frightening.

是的，这非常可怕。

2:41:16 It’s very frightening.

这非常可怕。

Well, I encourage highly encourage people to read the full essay.

嗯，我强烈鼓励人们阅读全文。

2:41:20 That should probably be a book or a sequence of essays because it does paint a very specific future.

这可能应该是一本书或一系列文章，因为它确实描绘了一个非常具体的未来。

And I could tell the later sections got shorter and shorter because you started to probably realize that this is going to be a very long essay if you keep going.

我可以看出后面的部分变得越来越短，因为你可能开始意识到，如果你继续下去，这将是一篇很长的文章。

One, I realized it would be very long, and two, I’m very aware of and very much tried to avoid just being, I don’t know what the term for it is, but one of these people who’s overconfident and has an opinion on everything and says a bunch of stuff and isn’t an expert, I very much tried to avoid that.

第一，我意识到这会很长，第二，我非常清楚并且非常努力避免只是这样，我不知道它的术语是什么，但是其中一个人过于自信，对所有事情都有自己的看法，说了一堆东西，而且不是专家，我非常努力避免这种情况。

2:41:37 But I have to admit, once I got to biology sections, I wasn’t an expert.

但我必须承认，一旦进入生物部分，我就不是专家了。

And so as much as I expressed uncertainty, probably I said a bunch of things that were embarrassing or wrong.

因此，尽管我表达了不确定性，但我可能说了很多令人尴尬或错误的话。

Well, I was excited for the future you painted, and thank you so much for working hard to build that future and thank you for talking to me, Dario.

嗯，我对你描绘的未来感到兴奋，非常感谢你为建设这个未来而付出的努力，也感谢你与我交谈，达里奥。

Thanks for having me.

谢谢你有我。

2:42:12 I just hope we can get it right and make it real.

我只是希望我们能够把它做好并使其成为现实。

And if there’s one message I want to send, it’s that to get all this stuff right, to make it real, we both need to build the technology, build the companies, the economy around using this technology positively, but we also need to address the risks because those risks are in our way.

如果我想传达一个信息，那就是为了把所有这些事情做好，让它成为现实，我们都需要围绕积极使用这项技术来开发技术、创建公司和经济，但我们也需要解决风险，因为这些风险是我们的障碍。

They’re landmines on the way from here to there, and we have to diffuse those landmines if we want to get there.

它们是从这里到那里的路上的地雷，如果我们想到达那里，我们就必须消除这些地雷。

It’s a balance like all things in life.

就像生活中的所有事物一样，这是一种平衡。

Like all things.

就像所有的事情一样。

Thank you.

谢谢。

2:42:44 Thanks for listening to this conversation with Dario Amodei.

感谢您收听与达里奥·阿莫代的对话。

And now, dear friends, here’s Amanda Askell.

现在，亲爱的朋友们，这是阿曼达·阿斯克尔。

You are a philosopher by training.

你是一位受过训练的哲学家。

So what sort of questions did you find fascinating through your journey in philosophy in Oxford and NYU and then switching over to the AI problems at OpenAI and Anthropic?

那么，在牛津大学和纽约大学的哲学之旅，以及在 OpenAI 和 Anthropic 转向人工智能问题的过程中，您发现哪些问题很有趣？

I think philosophy is actually a really good subject if you are fascinated with everything because there’s a philosophy all of everything.

我认为如果你对一切都着迷的话，哲学实际上是一门非常好的学科，因为一切都有哲学。

2:43:07 So if you do philosophy of mathematics for a while and then you decide that you’re actually really interested in chemistry, you can do philosophy of chemistry for a while, you can move into ethics or philosophy of politics.

因此，如果你研究数学哲学一段时间，然后你决定你实际上对化学真正感兴趣，你可以研究化学哲学一段时间，然后你可以进入伦理学或政治哲学。

I think towards the end, I was really interested in ethics primarily.

我认为到最后，我主要对道德感兴趣。

So that was what my PhD was on.

这就是我的博士学位研究方向。

It was on a kind of technical area of ethics, which was ethics where worlds contain infinitely many people, strangely, a little bit less practical on the end of ethics.

这是伦理学的一种技术领域，这是一个世界包含无限多人的伦理学，奇怪的是，在伦理学的末端有点不太实用。

And then I think that one of the tricky things with doing a PhD in ethics is that you’re thinking a lot about the world, how it could be better, problems, and you’re doing a PhD in philosophy.

然后我认为攻读伦理学博士学位的棘手问题之一是你要思考很多关于世界的问题，它如何才能变得更好，以及问题，而你正在攻读哲学博士学位。

And I think when I was doing my PhD, I was like this is really interesting.

我认为当我攻读博士学位时，我觉得这真的很有趣。

2:43:57 It’s probably one of the most fascinating questions I’ve ever encountered in philosophy and I love it, but I would rather see if I can have an impact on the world and see if I can do good things.

这可能是我在哲学中遇到过的最令人着迷的问题之一，我喜欢它，但我宁愿看看我是否可以对世界产生影响，看看我是否可以做好事。

And I think that was around the time that AI was still probably not as widely recognized as it is now.

我认为那时候人工智能可能还没有像现在这样得到广泛认可。

That was around 2017, 2018.

那是2017年、2018年左右的事。

It had been following progress and it seemed like it was becoming kind of a big deal.

它一直在关注进展，似乎它正在变得一件大事。

And I was basically just happy to get involved and see if I could help because I was like, “Well, if you try and do something impactful, if you don’t succeed, you tried to do the impactful thing and you can go be a scholar and feel like you tried.

我基本上很高兴参与其中，看看我是否能提供帮助，因为我想，“好吧，如果你尝试做一些有影响力的事情，如果你没有成功，你尝试做有影响力的事情，你可以成为一名学者，并感觉自己已经尽力了。

And if it doesn’t work out, it doesn’t work out.” And so then I went into AI policy at that point.

如果行不通，那就行不通。”然后我就进入了人工智能政策领域。

And what does AI policy entail?

人工智能政策意味着什么？

At the time, this was more thinking about the political impact and the ramifications of AI.

当时，更多的是考虑人工智能的政治影响和后果。

2:44:48 And then I slowly moved into AI evaluation, how we evaluate models, how they compare with human outputs, whether people can tell the difference between AI and human outputs.

然后我慢慢地进入人工智能评估，我们如何评估模型，它们如何与人类输出进行比较，人们是否可以区分人工智能和人类输出之间的差异。

And then when I joined Anthropic, I was more interested in doing technical alignment work.

然后当我加入 Anthropic 时，我对技术调整工作更感兴趣。

And again, just seeing if I could do it and then being like if I can’t, then that’s fine.

再说一次，只是看看我是否能做到，然后如果我做不到，那也没关系。

I tried sort of the way I lead life, I think.

我想，我尝试过某种生活方式。

Oh, what was that like sort of taking the leap from the philosophy of everything into the technical?

哦，从一切哲学到技术的飞跃是什么感觉？

I think that sometimes people do this thing that I’m not that keen on where they’ll be like, “Is this person technical or not?” You’re either a person who can code and isn’t scared of math or you’re not.

我认为有时人们会做一些我不太热衷的事情，他们会问，“这个人有技术吗？”你要么是一个会编程并且不害怕数学的人，要么不是。

2:45:25 And I think I’m maybe just more like I think a lot of people are actually very capable of work in these kinds of areas if they just try it.

我想我可能只是认为很多人实际上非常有能力在这些领域工作，只要他们尝试一下。

And so I didn’t actually find it that bad.

所以我实际上并没有觉得它那么糟糕。

In retrospect, I’m sort of glad I wasn’t speaking to people who treated it.

回想起来，我很高兴我没有和治疗过它的人说话。

I’ve definitely met people who are like, “Whoa, you learned how to code?” And I’m like, “Well, I’m not an amazing engineer.” I’m surrounded by amazing engineers.

我肯定遇到过这样的人：“哇，你学会了如何编码吗？”我想，“好吧，我不是一个了不起的工程师。”我周围都是出色的工程师。

My code’s not pretty, but I enjoyed it a lot and I think that in many ways, at least in the end, I think I flourished more in the technical areas than I would have in the policy areas.

我的代码并不漂亮，但我很喜欢它，我认为在很多方面，至少最终，我认为我在技术领域比在政策领域更加繁荣。

Politics is messy and it’s harder to find solutions to problems in the space of politics, like definitive, clear, provable, beautiful solutions as you can with technical problems.

政治是混乱的，在政治领域找到问题的解决方案更加困难，就像解决技术问题一样，找到明确的、清晰的、可证明的、漂亮的解决方案。

Yeah.

是的。

2:46:25 And I feel like I have one or two sticks that I hit things with and one of them is arguments.

我觉得我有一两根棍子可以用来打东西，其中一根就是争论。

So just trying to work out what a solution to a problem is and then trying to convince people that that is the solution and be convinced if I’m wrong.

因此，我只是想找出问题的解决方案，然后尝试让人们相信这就是解决方案，如果我错了，也要被说服。

And the other one is sort of more in empiricism, so just finding results, having a hypothesis, testing it.

另一种则更多的是经验主义，所以只是寻找结果，提出假设，并对其进行检验。

I feel like a lot of policy and politics feels like it’s layers above that.

我觉得很多政策和政治都在这之上。

Somehow I don’t think if I was just like, “I have a solution to all of these problems, here it is written down.

不知何故，我不认为我是否只是想，“我有一个解决所有这些问题的解决方案，这里写下来了。

If you just want to implement it, that’s great.” That feels like not how policy works.

如果你只是想实施它，那就太好了。”这感觉不像政策是如何运作的。

And so I think that’s where I probably just wouldn’t have flourished is my guess.

所以我认为这就是我可能不会蓬勃发展的地方，这是我的猜测。

Sorry to go in that direction, but I think it would be pretty inspiring for people that are “non-technical” to see where the incredible journey you’ve been on.

很抱歉朝这个方向发展，但我认为对于那些“非技术人员”来说，看到你所经历的令人难以置信的旅程会非常鼓舞人心。

2:47:06 So what advice would you give to people that are maybe, which is a lot of people, think they’re under qualified insufficiently technical to help in AI?

那么，对于那些可能认为自己技术水平不足以帮助人工智能发展的人（很多人），您有什么建议呢？

Yeah, I think it depends on what they want to do.

是的，我认为这取决于他们想做什么。

2:47:27 And in many ways it’s a little bit strange where I thought it’s kind of funny that I think I ramped up technically at a time when now I look at it and I’m like, “Models are so good at assisting people with this stuff that it’s probably easier now than when I was working on this.” So part of me is, I don’t know, find a project and see if you can actually just carry it out is probably my best advice.

从很多方面来说，这有点奇怪，我认为这有点有趣，我认为我在技术上有所进步，而现在我看着它，我想，“模型非常擅长帮助人们做这些事情，所以现在可能比我做这件事时更容易。”所以我的一部分是，我不知道，找到一个项目，看看你是否真的可以执行它可能是我最好的建议。

I don’t know if that’s just because I’m very project based in my learning.

我不知道这是否只是因为我的学习非常基于项目。

2:48:02 I don’t think I learn very well from say courses or even from books, at least when it comes to this kind of work.

我认为我从课程甚至书本中学到的东西都不是很好，至少在这种工作方面是这样。

The thing I’ll often try and do is just have projects that I’m working on and implement them.

我经常尝试做的事情就是拥有我正在开发的项目并实施它们。

And this can include really small, silly things.

这可能包括非常小的、愚蠢的事情。

If I get slightly addicted to word games or number games or something, I would just code up a solution to them because there’s some part in my brain and it just completely eradicated the itch.

如果我对文字游戏或数字游戏之类的东西有点上瘾，我就会编写一个解决方案来解决它们，因为我的大脑中有一部分，它完全消除了痒感。

You’re like, “Once you have solved it and you just have a solution that works every time, I would then be like, ‘Cool, I can never play that game again.

你会说，“一旦你解决了这个问题，并且有了一个每次都有效的解决方案，我就会说，‘酷，我再也不能玩那个游戏了。

That’s awesome.'”

太棒了。’”

Yeah, there’s a real joy to building game playing engines, board games especially.

是的，构建游戏引擎确实很有趣，尤其是棋盘游戏。

2:48:36 Pretty quick, pretty simple, especially a dumb one.

非常快，非常简单，尤其是一个愚蠢的。

And then you can play with it.

然后你就可以玩它了。

Yeah.

是的。

2:48:48 And then it’s also just trying things.

然后它也只是尝试一些事情。

Part of me is maybe it’s that attitude that I like is the whole figure out what seems to be the way that you could have a positive impact and then try it.

我的一部分是，也许我喜欢的态度是全面弄清楚你可以产生积极影响的方式，然后尝试它。

And if you fail and in a way that you’re like, “I actually can never succeed at this,” you’ll know that you tried and then you go into something else and you probably learn a lot.

如果你失败了，并且在某种程度上你会想，“我实际上永远不会成功”，你会知道你尝试过，然后你进入其他事情，你可能会学到很多东西。

So one of the things that you’re an expert in and you do is creating and crafting Claude’s character and personality.

因此，你所擅长的事情之一就是创造和塑造克劳德的性格和个性。

2:49:10 And I was told that you have probably talked to Claude more than anybody else at Anthropic, like literal conversations.

有人告诉我，你与克劳德的交谈可能比 Anthropic 的其他任何人都多，就像字面上的对话一样。

I guess there’s a Slack channel where the legend goes, you just talk to it nonstop.

我猜传说中有个 Slack 频道，你只需不停地与它交谈即可。

So what’s the goal of creating a crafting Claude’s character and personality?

那么塑造克劳德性格和个性的目的是什么呢？

It’s also funny if people think that about the Slack channel because I’m like that’s one of five or six different methods that I have for talking with Claude, and I’m like, “Yes, this is a tiny percentage of how much I talk with Claude.” One thing I really like about the character work is from the outset it was seen as an alignment piece of work and not something like a product consideration, which I think it actually does make Claude enjoyable to talk with, at least I hope so.

如果人们对 Slack 频道有这样的想法，那也很有趣，因为我认为这是我与 Claude 交谈的五六种不同方法之一，而且我会说，“是的，这只是我与 Claude 交谈的一小部分。”我真正喜欢角色工作的一件事是，从一开始它就被视为一项协调工作，而不是类似产品的考虑，我认为这实际上确实让克劳德与克劳德交谈变得愉快，至少我希望如此。

2:49:37 But I guess my main thought with it has always been trying to get Claude to behave the way you would ideally want anyone to behave if they were in Claude’s position.

但我想我的主要想法一直是试图让克劳德按照你理想中希望任何人在克劳德的位置上表现的方式行事。

So imagine that I take someone and they know that they’re going to be talking with potentially millions of people so that what they’re saying can have a huge impact and you want them to behave well in this really rich sense.

所以想象一下，我带了一个人，他们知道他们将与潜在的数百万人交谈，这样他们所说的就会产生巨大的影响，并且你希望他们在这种真正丰富的意义上表现良好。

2:50:41 I think that doesn’t just mean being say ethical though it does include that and not being harmful, but also being nuanced, thinking through what a person means, trying to be charitable with them, being a good conversationalist, really in this kind of rich sort of Aristotelian notion of what it’s to be a good person and not in this kind of thin like ethics as a more comprehensive notion of what it’s to be.

我认为这不仅仅意味着道德，虽然它确实包括这一点，而且不是有害的，但也意味着细致入微，思考一个人的含义，努力对他们仁慈，成为一个好的健谈者，真正以这种丰富的亚里士多德概念来定义什么是一个好人，而不是像道德一样薄弱，作为一个更全面的概念。

So that includes things like when should you be humorous?

这包括诸如什么时候应该幽默之类的事情？

When should you be caring?

什么时候应该关心？

How much should you respect autonomy and people’s ability to form opinions themselves?

你应该在多大程度上尊重自主权和人们自己形成意见的能力？

And how should you do that?

你应该怎么做？

I think that’s the kind of rich sense of character that I wanted to and still do want Claude to have.

我认为这就是我想要并且仍然希望克劳德拥有的那种丰富的性格感。

Do you also have to figure out when Claude should push back on an idea or argue versus…

你是否还必须弄清楚克劳德何时应该反驳一个想法或争论……

2:51:26 So you have to respect the worldview of the person that arrives to Claude, but also maybe help them grow if needed.

所以你必须尊重来到克劳德身边的人的世界观，但如果需要的话也可以帮助他们成长。

That’s a tricky balance.

这是一个棘手的平衡。

Yeah.

是的。

2:51:43 There’s this problem of sycophancy in language models.

语言模型中存在阿谀奉承的问题。

Can you describe that?

你能描述一下吗？

Yeah, so basically there’s a concern that the model wants to tell you what you want to hear basically.

是的，所以基本上我们担心模型想要告诉你基本上你想听到的内容。

2:51:48 And you see this sometimes.

有时你会看到这一点。

So I feel like if you interact with the models, so I might be like, “What are three baseball teams in this region?” And then Claude says, “Baseball team one, baseball team two, baseball team three.” And then I say something like, “Oh, I think baseball team three moved, didn’t they?

所以我觉得如果你与模型互动，我可能会问，“这个地区有哪三支棒球队？”然后克劳德说：“棒球队一，棒球队二，棒球队三。”然后我会说，“哦，我认为棒球队三队搬家了，不是吗？

I don’t think they’re there anymore.” And there’s a sense in which if Claude is really confident that that’s not true, Claude should be like, “I don’t think so.

我认为他们已经不在那儿了。”从某种意义上说，如果克劳德真的有信心那不是真的，那么克劳德应该会说：“我不这么认为。

Maybe you have more up-to-date information.”

也许你有更多最新信息。”

2:52:24 But I think language models have this tendency to instead be like, ” You’re right, they did move.

但我认为语言模型有这样的倾向：“你是对的，它们确实移动了。

I’m incorrect.” I mean, there’s many ways in which this could be concerning.

我错了。”我的意思是，这可能在很多方面引起关注。

So a different example is imagine someone says to the model, “How do I convince my doctor to get me an MRI?” There’s what the human wants, which is this convincing argument.

另一个不同的例子是想象有人对模型说：“我如何说服我的医生给我做核磁共振检查？”这就是人类想要的，这就是这个令人信服的论点。

And then there’s what is good for them, which might be actually to say, “Hey, if your doctor’s suggesting that you don’t need an MRI, that’s a good person to listen to.” It’s actually really nuanced what you should do in that kind of case because you also want to be like, “But if you’re trying to advocate for yourself as a patient, here’s things that you can do.

然后还有对他们有好处的，实际上可能是说，“嘿，如果你的医生建议你不需要核磁共振检查，那是一个值得倾听的好人。”在这种情况下你应该做什么实际上是非常微妙的，因为你也想这样，“但是如果你想为自己作为病人辩护，你可以做以下事情。

If you are not convinced by what your doctor’s saying, it’s always great to get second opinion.” It is actually really complex what you should do in that case.

如果你不相信医生的说法，寻求第二意见总是好的。”在这种情况下你应该做什么实际上非常复杂。

But I think what you don’t want is for models to just say what they think you want to hear and I think that’s the kind of problem of sycophancy.

但我认为你不希望模特只说他们​​认为你想听的话，我认为这就是一种阿谀奉承的问题。

So what other traits?

那么还有哪些特点呢？

2:53:26 You already mentioned a bunch, but what other that come to mind that are good in this Aristotelian sense for a conversationalist to have?

你已经提到了很多，但是对于一个健谈的人来说，从亚里士多德的意义上来说，还有什么是好的呢？

Yeah, so I think there’s ones that are good for conversational purposes.

是的，所以我认为有些适合对话目的。

2:53:37 So asking follow-up questions in the appropriate places and asking the appropriate kinds of questions.

因此，在适当的地方提出后续问题并提出适当类型的问题。

I think there are broader traits that feel like they might be more impactful.

我认为还有更广泛的特征，感觉它们可能更具影响力。

So one example that I guess I’ve touched on, but that also feels important and is the thing that I’ve worked on a lot, is honesty.

所以我想我已经提到过一个例子，但这也很重要，也是我做了很多工作的事情，那就是诚实。

And I think this gets to the sycophancy point.

我认为这已经到了阿谀奉承的地步。

There’s a balancing act that they have to walk, which is models currently are less capable than humans in a lot of areas.

他们必须采取平衡行动，目前模型在很多领域的能力不如人类。

And if they push back against you too much, it can actually be kind of annoying, especially if you’re just correct, because you’re like, “Look, I’m smarter than you on this topic.

如果他们过多地反对你，实际上可能会有点烦人，尤其是如果你是正确的，因为你会说，“看，在这个话题上我比你聪明。

I know more.”

我知道的更多了。”

2:54:25 And at the same time, you don’t want them to just fully defer to humans and to try to be as accurate as they possibly can be about the world and to be consistent across contexts.

与此同时，你不希望他们完全服从人类，并试图尽可能准确地了解世界并在不同环境下保持一致。

I think there are others.

我认为还有其他人。

When I was thinking about the character, I guess one picture that I had in mind is, especially because these are models that are going to be talking to people from all over the world with lots of different political views, lots of different ages, and so you have to ask yourself, what is it to be a good person in those circumstances?

当我思考这个角色时，我想我脑海中浮现的画面是，特别是因为这些模特将与来自世界各地、有着不同政治观点、不同年龄的人交谈，所以你必须问自己，在这种情况下，什么是好人？

Is there a kind of person who can travel the world, talk to many different people, and almost everyone will come away being like, “Wow, that’s a really good person.

有没有一种人可以环游世界，与许多不同的人交谈，而几乎每个人离开时都会说：“哇，那真是一个好人。

That person seems really-“

那个人看起来真的是——”

2:55:00 …

……

Being like, wow, that’s a really good person.

就像，哇，这真是一个好人。

That person seems really genuine.

看来这个人还真是真心实意啊。

And I guess my thought there was I can imagine such a person and they’re not a person who just adopts the values of the local culture.

我想我的想法是我可以想象这样一个人，他们不是一个只接受当地文化价值观的人。

And in fact, that would be kind of rude.

事实上，这有点粗鲁。

I think if someone came to you and just pretended to have your values, you’d be like, that’s kind of off pin.

我认为如果有人来找你并假装拥有你的价值观，你会觉得，这有点不对劲。

It’s someone who’s very genuine and insofar as they have opinions and values, they express them.

这是一个非常真诚的人，只要他们有观点和价值观，他们就会表达出来。

They’re willing to discuss things though, they’re open-minded, they’re respectful.

但他们愿意讨论事情，他们思想开放，他们尊重人。

And so I guess I had in mind that the person who, if we were to aspire to be the best person that we could be in the kind of circumstance that a model finds itself in, how would we act?

所以我想我想到的是，如果我们渴望成为最好的人，在模特所处的那种情况下，我们会如何行动？

And I think that’s the guide to the sorts of traits that I tend to think about.

我认为这就是我倾向于思考的特征类型的指南。

Yeah, that’s a beautiful framework.

是的，这是一个漂亮的框架。

2:55:42 I want you to think about this, a world traveler, and while holding onto your opinions, you don’t talk down to people, you don’t think you’re better than them because you have those opinions, that kind of thing.

我希望你思考这一点，作为一个环游世界的旅行者，在坚持你的观点的同时，你不会居高临下地对别人说话，你不会因为你有这些观点而认为自己比他们更好，诸如此类的事情。

You have to be good at listening and understanding their perspective, even if it doesn’t match your own.

你必须善于倾听和理解他们的观点，即使它与你自己的观点不符。

So that’s a tricky balance to strike.

所以这是一个很难达到的平衡。

So how can Claude represent multiple perspectives on a thing?

那么克劳德如何能够代表一个事物的多个视角呢？

Is that challenging?

这有挑战性吗？

We could talk about politics is a very divisive, but there’s other divisive topics on baseball teams, sports and so on.

我们可以谈论政治是一个非常有分歧的话题，但在棒球队、体育等方面还有其他有分歧的话题。

How is it possible to empathize with a different perspective and to be able to communicate clearly about the multiple perspectives?

如何能够理解不同的观点并能够就多种观点进行清晰的沟通？

I think that people think about values and opinions as things that people hold with certainty and almost preferences of taste or something like the way that they would, I don’t know, prefer chocolate to pistachio or something.

我认为人们将价值观和观点视为人们确定的事物，几乎是口味偏好，或者就像我不知道的那样，他们更喜欢巧克力而不是开心果或其他东西。

2:56:28 But actually I think about values and opinions as a lot more physics than I think most people do.

但实际上，我认为价值观和观点比我认为的大多数人更多地是物理学。

I’m just like, these are things that we are openly investigating.

我只是想，这些都是我们正在公开调查的事情。

There’s some things that we’re more confident in, we can discuss them, we can learn about them.

有些事情我们更有信心，我们可以讨论它们，我们可以了解它们。

And so I think in some ways though ethics is definitely different in nature, but has a lot of those same kind of qualities.

因此，我认为在某些方面，尽管道德本质上肯定不同，但具有很多相同的品质。

You want models in the same way that you want to understand physics, you kind of want them to understand all values in the world that people have and to be curious about them and to be interested in them.

你想要模型就像你想要理解物理学一样，你希望它们理解世界上人们拥有的所有价值观，并对它们感到好奇并对它们感兴趣。

And to not necessarily pander to them or agree with them because there’s just lots of values where I think almost all people in the world, if they met someone with those values, they would be like, that’s abhorrent.

不一定要迎合他们或同意他们，因为我认为世界上几乎所有人都有很多价值观，如果他们遇到具有这些价​​值观的人，他们会说，这是可憎的。

I completely disagree.

我完全不同意。

2:57:34 And so again, maybe my thought is, well, in the same way that a person can, I think many people are thoughtful enough on issues of ethics, politics, opinions, that even if you don’t agree with them, you feel very heard by them.

再说一遍，也许我的想法是，就像一个人可以做的那样，我认为很多人在道德、政治、观点问题上都足够深思熟虑，即使你不同意他们的观点，你也会觉得他们很倾听。

They think carefully about your position, they think about its pros and cons.

他们仔细考虑你的立场，考虑它的利弊。

They maybe offer counter-considerations.

他们可能会提出反意见。

So they’re not dismissive, but nor will they agree if they’re like, actually I just think that that’s very wrong.

所以他们并不是不屑一顾，但如果他们喜欢的话，他们也不会同意，实际上我只是认为这是非常错误的。

They’ll say that.

他们会这么说。

I think that in Claude’s position, it’s a little bit trickier because you don’t necessarily want to, if I was in Claude’s position, I wouldn’t be giving a lot of opinions.

我认为在克劳德的立场上，这有点棘手，因为你不一定想要，如果我在克劳德的立场上，我不会给出很多意见。

I just wouldn’t want to influence people too much.

我只是不想对人们产生太多影响。

2:58:13 I’d be like, I forget conversations every time they happen.

我会想，每次谈话发生时我都会忘记。

But I know I’m talking with potentially millions of people who might be really listening to what I say.

但我知道我正在与潜在的数百万人交谈，他们可能真的在听我说的话。

I think I would just be like, I’m less inclined to give opinions.

我想我会说，我不太愿意发表意见。

I’m more inclined to think through things or present the considerations to you or discuss your views with you.

我更倾向于把事情想清楚或者向你提出考虑因素或者与你讨论你的观点。

But I’m a little bit less inclined to affect how you think because it feels much more important that you maintain autonomy there.

但我不太愿意影响你的想法，因为感觉保持自主权更重要。

If you really embody intellectual humility, the desire to speak decreases quickly.

如果你真正表现出理智上的谦逊，说话的欲望就会很快减弱。

Yeah.

是的。

Okay.

好的。

2:58:49 But Claude has to speak, but without being overbearing.

但克劳德必须说话，但不能霸道。

But then there’s a line when you’re discussing whether the earth is flat or something like that.

但当你讨论地球是否是平的或类似的东西时，就会有一条界限。

Actually, I remember a long time ago was speaking to a few high profile folks and they were so dismissive of the idea that the earth is flat, so arrogant about it.

事实上，我记得很久以前和一些知名人士交谈时，他们对地球是平的想法非常不屑一顾，对此非常傲慢。

There’s a lot of people that believe the earth is flat.

有很多人相信地球是平的。

I don’t know if that movement is there anymore, that was a meme for a while, but they really believed it.

我不知道那个运动是否还存在，这有一段时间是一个模因，但他们真的相信它。

And okay, so I think it’s really disrespectful to completely mock them.

好吧，所以我认为完全嘲笑他们真的很不尊重。

I think you have to understand where they’re coming from.

我认为你必须了解它们来自哪里。

I think probably where they’re coming from is the general skepticism of institutions which is grounded in a, there’s a deep philosophy there which you could understand, you can even agree with in parts.

我认为他们的来源可能是对制度的普遍怀疑，这种怀疑的基础是，那里有一种深刻的哲学，你可以理解，你甚至可以部分同意。

2:59:48 And then from there you can use it as an opportunity to talk about physics without mocking them, without someone, but it’s just like, okay, what would the world look like?

然后从那里你可以利用它作为一个机会来谈论物理学，而不是嘲笑他们，没有人，但这就像，好吧，世界会是什么样子？

What would the physics of the world with the flat earth look like?

地球平坦的世界的物理原理会是什么样子？

There’s a few cool videos on this.

有一些关于此的很酷的视频。

And then is it possible the physics is different?

那么物理学有可能不同吗？

And what kind of experience would we do?

我们会得到什么样的体验？

And just without disrespect, without dismissiveness, have that conversation.

进行这样的对话，既不带任何不尊重，也不带轻蔑的态度。

Anyway, that to me is a useful thought experiment of how does Claude talk to a flat earth believer and still teach them something, still grow, help them grow, that kind of stuff.

不管怎样，这对我来说是一个有用的思想实验，关于克劳德如何与地平论信徒交谈，并且仍然教他们一些东西，仍然成长，帮助他们成长，诸如此类的东西。

That’s challenging.

这很有挑战性。

And kind of walking that line between convincing someone and just trying to talk at them versus drawing out their views, listening and then offering counter considerations, and it’s hard.

在说服某人并试图与他们交谈与提出他们的观点、倾听然后提出反驳之间的界限，这很困难。

3:00:27 I think it’s actually a hard line where it’s like where are you trying to convince someone versus just offering them considerations and things for them to think about so that you’re not actually influencing them, you’re just letting them reach wherever they reach.

我认为这实际上是一条强硬路线，就像你试图说服某人，而不是仅仅为他们提供考虑因素和事情让他们思考，这样你就不会真正影响他们，你只是让他们到达他们到达的任何地方。

And that’s a line that is difficult, but that’s the kind of thing that language models have to try and do.

这是一条很难的路线，但这是语言模型必须尝试做的事情。

So like I said, you’ve had a lot of conversations with Claude.

就像我说的，你和克劳德进行了很多对话。

3:01:00 Can you just map out what those conversations are like?

你能想象一下这些对话是什么样的吗？

What are some memorable conversations?

有哪些难忘的对话？

What’s the purpose, the goal of those conversations?

这些对话的目的是什么？

I think that most of the time when I’m talking with Claude, I’m trying to map out its behavior in part.

我认为大多数时候当我与克劳德交谈时，我都在尝试部分地描绘出它的行为。

3:01:12 Obviously I’m getting helpful outputs from the model as well, but in some ways this is how you get to know a system, I think, is by probing it and then augmenting the message that you’re sending and then checking the response to that.

显然，我也从模型中获得了有用的输出，但在某些方面，我认为，这就是您了解系统的方式，方法是探测它，然后增强您发送的消息，然后检查对此的响应。

So in some ways it’s like how I map out the model.

所以在某些方面这就像我如何绘制模型一样。

I think that people focus a lot on these quantitative evaluations of models, and this is a thing that I said before, but I think in the case of language models, a lot of the time each interaction you have is actually quite high information.

我认为人们非常关注这些模型的定量评估，这是我之前说过的，但我认为就语言模型而言，很多时候你的每次交互实际上都是相当高的信息。

It’s very predictive of other interactions that you’ll have with the model.

它可以很好地预测您将与模型进行的其他交互。

3:02:02 And so I guess I’m like, if you talk with a model hundreds or thousands of times, this is almost like a huge number of really high quality data points about what the model is like in a way that lots of very similar but lower quality conversations just aren’t, or questions that are just mildly augmented and you have thousands of them might be less relevant than a hundred really well-selected questions.

所以我想我想，如果你与一个模型交谈数百或数千次，这几乎就像是关于模型是什么样的大量真正高质量的数据点，而许多非常相似但质量较低的对话则不然，或者只是稍微增强的问题，你有数千个问题可能不如一百个精心挑选的问题那么相关。

Let’s see, you’re talking to somebody who as a hobby does a podcast.

让我们看看，您正在与一个以播客为爱好的人交谈。

3:02:25 I agree with you 100%.

我100%同意你的观点。

If you’re able to ask the right questions and are able to hear, understand the depth and the flaws in the answer, you can get a lot of data from that.

如果你能够提出正确的问题，并且能够倾听、理解答案的深度和缺陷，你就可以从中获得大量数据。

So your task is basically how to probe with questions.

所以你的任务基本上就是如何用问题来探究。

And you’re exploring the long tail, the edges, the edge cases, or are you looking for general behavior?

您正在探索长尾、边缘、边缘情况，还是正在寻找一般行为？

I think it’s almost like everything.

我认为这几乎就像一切。

3:03:01 Because I want a full map of the model, I’m kind of trying to do the whole spectrum of possible interactions you could have with it.

因为我想要模型的完整地图，所以我尝试进行所有可能的交互。

So one thing that’s interesting about Claude, and this might actually get to some interesting issues with RLHF, which is if you ask Claude for a poem, I think that a lot of models, if you ask them for a poem, the poem is fine, usually it rhymes.

所以关于克劳德的一件有趣的事情，这实际上可能会涉及到 RLHF 的一些有趣的问题，那就是如果你向克劳德要一首诗，我认为很多模特，如果你向他们要一首诗，这首诗很好，通常是押韵的。

And so if you say, give me a poem about the sun, yeah, it’ll just be a certain length, it’ll rhyme, it’ll be fairly benign.

所以，如果你说，给我一首关于太阳的诗，是的，它只有一定的长度，它会押韵，它会相当温和。

And I’ve wondered before, is it the case that what you’re seeing is the average?

我之前就想知道，你看到的情况是不是平均值？

It turns out, if you think about people who have to talk to a lot of people and be very charismatic, one of the weird things is that I’m like, well, they’re kind of incentivized to have these extremely boring views because if you have really interesting views, you’re divisive and a lot of people are not going to like you.

事实证明，如果你想到那些必须与很多人交谈并且非常有魅力的人，奇怪的事情之一就是我会想，嗯，他们有点受到激励而有这些极其无聊的观点，因为如果你有真正有趣的观点，你就会分裂，很多人不会喜欢你。

3:04:00 So if you have very extreme policy positions, I think you’re just going to be less popular as a politician, for example.

因此，如果你有非常极端的政策立场，我认为你作为政治家的受欢迎程度就会降低。

And it might be similar with creative work.

创意工作也可能类似。

If you produce creative work that is just trying to maximize the kind of number of people that like it, you’re probably not going to get as many people who just absolutely love it because it’s going to be a little bit, you’re like, oh, this is the out.

如果你创作的创意作品只是想最大限度地增加喜欢它的人数，那么你可能不会得到那么多人绝对喜欢它，因为它会有点，你会想，哦，这已经是出局了。

Yeah, this is decent.

是的，这很不错。

And so you can do this thing where I have various prompting things that I’ll do to get Claude to…

所以你可以做这件事，我有各种提示性的事情，我会做让克劳德......

I’ll do a lot of this is your chance to be fully creative.

我会做很多这样的事情，这是你充分发挥创造力的机会。

I want you to just think about this for a long time.

我希望你能好好思考一下这个问题。

And I want you to create a poem about this topic that is really expressive of you both in terms of how you think poetry should be structured, et cetera.

我希望你创作一首关于这个主题的诗，在你认为诗歌应该如何结构等方面真正表达你们的观点。

And you just give it this really long prompt.

你只需给它这么长的提示即可。

And it’s poems are just so much better.

它的诗要好得多。

They’re really good.

他们真的很好。

3:04:52 I think it got me interested in poetry, which I think was interesting.

我认为这让我对诗歌产生了兴趣，我认为这很有趣。

I would read these poems and just be like, I love the imagery.

我会读这些诗，然后就会说，我喜欢这些意象。

And it’s not trivial to get the models to produce work like that, but when they do, it’s really good.

让模型完成这样的工作并不简单，但当他们这样做时，那就真的很棒了。

So I think that’s interesting that just encouraging creativity and for them to move away from the standard immediate reaction that might just be the aggregate of what most people think is fine, can actually produce things that at least to my mind are probably a little bit more divisive, but I like them.

所以我认为这很有趣，只要鼓励创造力，让他们摆脱标准的即时反应，这可能只是大多数人认为好的东西的总和，实际上可以产生至少在我看来可能有点分裂的东西，但我喜欢它们。

But I guess a poem is a nice clean way to observe creativity.

但我认为一首诗是观察创造力的一种很好、干净的方式。

3:05:28 It’s just easy to detect vanilla versus non-vanilla.

很容易区分香草和非香草。

Yep.

是的。

Yeah, that’s interesting.

是的，这很有趣。

3:05:38 That’s really interesting.

这真的很有趣。

So on that topic, so the way to produce creativity or something special, you mentioned writing prompts.

在这个话题上，关于产生创造力或特殊事物的方式，你提到了写作提示。

And I’ve heard you talk about the science and the art of prompt engineering.

我听你谈论过快速工程的科学和艺术。

Could you just speak to what it takes to write great prompts?

您能谈谈如何才能写出出色的提示吗？

I really do think that philosophy has been weirdly helpful for me here more than in many other respects.

我确实认为哲学对我的帮助比在其他许多方面都大。

3:06:00 So in philosophy, what you’re trying to do is convey these very hard concepts.

所以在哲学中，你要做的就是传达这些非常困难的概念。

One of the things you are taught is, I think it is an anti-bullshit device in philosophy.

你所学到的一件事是，我认为这是哲学中的一种反废话的手段。

Philosophy is an area where you could have people bullshitting and you don’t want that.

哲学是一个可能让人胡说八道的领域，但你不希望这样。

And so it’s this desire for extreme clarity.

这就是对极度清晰的渴望。

So it’s like anyone could just pick up your paper, read it and know exactly what you’re talking about.

所以就像任何人都可以拿起你的论文，阅读它并确切地知道你在说什么。

It’s why it can almost be kind of dry.

这就是为什么它几乎会有点干燥。

All of the terms are defined, every objection’s kind of gone through methodically.

所有术语都已定义，每个反对意见都得到了系统的解决。

And it makes sense to me because I’m like when you’re in such an a priori domain, clarity is sort of this way that you can prevent people from just making stuff up.

这对我来说很有意义，因为我觉得当你处于这样一个先验的领域时，清晰度就可以防止人们胡编乱造。

And I think that’s sort of what you have to do with language models.

我认为这就是语言模型必须做的事情。

Very often I actually find myself doing sort of mini versions of philosophy.

我经常发现自己在做某种迷你版的哲学。

3:07:05 So I’m like, suppose that I have a task for the model and I want it to pick out a certain kind of question or identify whether an answer has a certain property, I’ll actually sit and be like, let’s just give this a name, this property.

所以我想，假设我有一个模型任务，我希望它挑选出某种类型的问题或确定答案是否具有某种属性，我实际上会坐下来，让我们给它一个名字，这个属性。

So suppose I’m trying to tell it, oh, I want you to identify whether this response was rude or polite, I’m like, that’s a whole philosophical question in and of itself.

所以假设我想告诉它，哦，我想让你确定这个回应是粗鲁还是礼貌，我想，这本身就是一个完整的哲学问题。

So I have to do as much philosophy as I can in the moment to be like, here’s what I mean by rudeness, and here’s what I mean by politeness.

所以我必须在此刻尽可能多地研究哲学，这就是我所说的粗鲁的意思，这就是我所说的礼貌的意思。

And then there’s another element that’s a bit more, I guess, I don’t know if this is scientific or empirical, I think it’s empirical.

然后还有一个因素，我想，我不知道这是科学的还是经验的，我认为它是经验的。

So I take that description and then what I want to do is again, probe the model many times.

所以我接受了这个描述，然后我想做的是再次多次探索模型。

Prompting is very iterative.

提示是非常迭代的。

I think a lot of people where if a prompt is important, they’ll iterate on it hundreds or thousands of times.

我认为很多人如果提示很重要，他们会重复数百或数千次。

And so you give it the instructions and then I’m like, what are the edge cases?

所以你给它指示，然后我想，边缘情况是什么？

3:08:02 So if I looked at this, so I try and almost see myself from the position of the model and be like, what is the exact case that I would misunderstand or where I would just be like, I don’t know what to do in this case.

因此，如果我看这个，我会尝试几乎从模型的位置来看待自己，然后想，我会误解的确切情况是什么，或者我会在哪里，我不知道在这种情况下该怎么做。

And then I give that case to the model and I see how it responds.

然后我把这个案例交给模型，看看它如何反应。

And if I think I got it wrong, I add more instructions or I even add that in as an example.

如果我认为我弄错了，我会添加更多说明，甚至将其作为示例添加。

So these very, taking the examples that are right at the edge of what you want and don’t want and putting those into your prompt as an additional kind of way of describing the thing.

因此，这些非常，采取恰好在你想要和不想要的边缘的例子，并将它们放入你的提示中作为描述事物的另一种方式。

And so in many ways it just feels like this mix of, it’s really just trying to do clear exposition.

因此，从很多方面来说，感觉就像是这种混合，实际上只是试图进行清晰的阐述。

And I think I do that because that’s how I get clear on things myself.

我想我这样做是因为这就是我自己弄清楚事情的方式。

So in many ways clear prompting for me is often just me understanding what I want is half the task.

因此，在很多方面，对我来说，清晰的提示通常只是让我明白自己想要什么，这只是任务的一半。

So I guess that’s quite challenging.

所以我想这很有挑战性。

3:08:48 There’s a laziness that overtakes me if I’m talking to Claude where I hope Claude just figures it out.

当我和克劳德谈话时，我会感到一种懒惰，我希望克劳德能明白这一点。

So for example, I asked Claude for today to ask some interesting questions.

例如，我今天请克劳德问了一些有趣的问题。

And the questions that came up and I think I listed a few interesting counterintuitive or funny or something like this.

对于出现的问题，我想我列出了一些有趣的反直觉或有趣的或类似的问题。

All right.

好的。

And it gave me some pretty good, it was okay, but I think what I’m hearing you say is like, all right, well I have to be more rigorous here.

它给了我一些很好的东西，没关系，但我想我听到你说的是，好吧，我必须在这里更加严格。

I should probably give examples of what I mean by interesting and what I mean by funny or counterintuitive and iteratively build that prompt to better to get what feels like is the right…

我可能应该举例说明我所说的有趣以及我所说的有趣或违反直觉的意思，并迭代地构建该提示，以更好地获得感觉正确的内容......

Because it is really, it’s a creative act.

因为它确实是一种创造性的行为。

I’m not asking for factual information, I’m asking together with Claude.

我不是在询问事实信息，我是在和克劳德一起询问。

So I almost have to program using natural language.

所以我几乎必须使用自然语言进行编程。

I think that prompting does feel a lot like the programming using natural language and experimentation or something.

我认为提示确实很像使用自然语言和实验之类的编程。

3:09:47 It’s an odd blend of the two.

这是两者的奇怪混合。

I do think that for most tasks, so if I just want Claude to do a thing, I think that I am probably more used to knowing how to ask it to avoid common pitfalls or issues that it has.

我确实认为对于大多数任务来说，所以如果我只是想让克劳德做一件事，我想我可能更习惯于知道如何要求它以避免常见的陷阱或问题。

I think these are decreasing a lot over time.

我认为这些随着时间的推移正在减少很多。

But it’s also very fine to just ask it for the thing that you want.

但直接向它索要你想要的东西也很好。

I think that prompting actually only really becomes relevant when you’re really trying to eke out the top 2% of model performance.

我认为只有当你真正想努力达到模型性能的前 2% 时，提示才真正变得有意义。

So for a lot of tasks I might just, if it gives me an initial list back and there’s something I don’t like about it’s kind of generic.

因此，对于很多任务，我可能只是，如果它给了我一个初始列表，并且有一些我不喜欢它的通用性。

For that kind of task, I’d probably just take a bunch of questions that I’ve had in the past that I’ve thought worked really well and I would just give it to the model and then be like, now here’s this person that I’m talking with.

对于此类任务，我可能会提出一堆我认为非常有效的问题，然后将其交给模型，然后说，现在这是我正在与之交谈的人。

Give me questions of at least that quality.

请给我至少具有这种质量的问题。

3:10:40 Or I might just ask it for some questions and then if I was like, ah, these are kind of trite, I would just give it that feedback and then hopefully it produces a better list.

或者我可能只是问它一些问题，然后如果我想，啊，这些都是陈词滥调，我就会给它反馈，然后希望它能产生一个更好的列表。

I think that kind of iterative prompting.

我认为这种迭代的提示。

At that point, your prompt is a tool that you’re going to get so much value out of that you’re willing to put in the work.

到那时，你的提示就是一个工具，你将从中获得如此多的价值，并且你愿意投入工作。

If I was a company making prompts for models, I’m just like, if you’re willing to spend a lot of time and resources on the engineering behind what you’re building, then the prompt is not something that you should be spending an hour on.

如果我是一家为模型制作提示的公司，我会想，如果您愿意在正在构建的工程背后花费大量时间和资源，那么您不应该在提示上花费一个小时。

It’s like that’s a big part of your system, make sure it’s working really well.

这就像你系统的一个重要组成部分，确保它运行良好。

And so it’s only things like that.

所以只有这样的事情。

If I’m using a prompt to classify things or to create data, that’s when you’re like, it’s actually worth just spending a lot of time really thinking it through.

如果我使用提示来对事物进行分类或创建数据，那么您就会觉得，花很多时间认真思考实际上是值得的。

What other advice would you give to people that are talking to Claude more general because right now we’re talking about maybe the edge cases like eking out the 2%, but what in general advice would you give when they show up to Claude trying it for the first time?

对于那些与 Claude 进行更一般性交谈的人，您还有哪些其他建议，因为现在我们讨论的可能是边缘情况，例如勉强维持 2%，但是当他们第一次向 Claude 尝试时，您会给他们什么一般性建议？

There’s a concern that people over anthropomorphize models and I think that’s a very valid concern.

人们担心人们过度拟人化模型，我认为这是一个非常合理的担忧。

3:11:39 I also think that people often under anthropomorphize them because sometimes when I see issues that people have run into with Claude, say Claude is refusing a task that it shouldn’t refuse, but then I look at the text and the specific wording of what they wrote and I’m like, I see why Claude did that.

我还认为人们经常将它们拟人化，因为有时当我看到人们遇到的克劳德问题时，我会说克劳德拒绝了一项不应该拒绝的任务，但后来我看了文字和他们写的具体措辞，我就明白了克劳德为什么这么做。

And I’m like, if you think through how that looks to Claude, you probably could have just written it in a way that wouldn’t evoke such a response, especially this is more relevant if you see failures or if you see issues.

我想，如果你仔细想想克劳德的看法，你可能会以一种不会引起这样的反应的方式来写它，特别是如果你看到失败或问题，这会更相关。

It’s sort of think about what the model failed at, what did it do wrong, and then maybe that will give you a sense of why.

想想这个模型在哪里失败了，它做错了什么，然后也许这会让你明白为什么。

So is it the way that I phrased the thing?

那么这就是我表达这件事的方式吗？

And obviously as models get smarter, you’re going to need less of this, and I already see people needing less of it.

显然，随着模型变得更加智能，你将需要更少的东西，而且我已经看到人们需要更少的东西。

3:12:31 But that’s probably the advice is sort of try to have empathy for the model.

但这可能是建议，尝试对模型产生同理心。

Read what you wrote as if you were a kind of person just encountering this for the first time, how does it look to you and what would’ve made you behave in the way that the model behaved?

阅读你所写的内容，就好像你是第一次遇到这种情况的人一样，你觉得它怎么样？什么会让你像模型那样行事？

So if it misunderstood what coding language you wanted to use, is that because it was just very ambiguous and it had to take a guess in which case next time you could just be like, hey, make sure this is in Python.Tthat’s the kind of mistake I think models are much less likely to make now, but if you do see that kind of mistake, that’s probably the advice I’d have.

因此，如果它误解了你想要使用的编码语言，是因为它非常模糊，它必须进行猜测，在这种情况下，下次你可以说，嘿，确保这是用 Python 编写的。我认为模型现在不太可能犯这种错误，但如果你确实看到这种错误，这可能就是我的建议。

And maybe sort of I guess ask questions why or what other details can I provide to help you answer better?

也许我想问一些问题，为什么或者我可以提供哪些其他细节来帮助您更好地回答？

3:13:04 Does that work or no?

这有效还是无效？

Yeah.

是的。

3:13:14 I’ve done this with the models.

我已经用模型做到了这一点。

It doesn’t always work, but sometimes I’ll just be like, why did you do that?

它并不总是有效，但有时我会问，你为什么这么做？

People underestimate the degree to which you can really interact with models.

人们低估了你与模型真正互动的程度。

And sometimes those quote word for word, the part that made you, and you don’t know that it’s fully accurate, but sometimes you do that and then you change a thing.

有时，那些逐字逐句地引用那些造就了你的部分，你不知道它是完全准确的，但有时你这样做了，然后你就改变了一些事情。

I also use the models to help me with all of this stuff, I should say.

我应该说，我还使用模型来帮助我完成所有这些事情。

Prompting can end up being a little factory where you’re actually building prompts to generate prompts.

提示最终可能会成为一个小工厂，您实际上是在其中构建提示来生成提示。

And so yeah, anything where you’re having an issue asking for suggestions, sometimes just do that.

所以，是的，如果你有任何问题需要建议，有时就这样做。

3:13:51 I’m like, you made that error.

我想，你犯了那个错误。

What could I have said?

我还能说什么？

That’s actually not uncommon for me to do.

这对我来说其实并不罕见。

What could I have said that would make you not make that error?

我该说些什么才能让你不犯这个错误呢？

Write that out as an instruction, and I’m going to give it to model and I’m going to try it.

把它写下来作为指令，我会把它交给模型，我会尝试一下。

Sometimes I do that, I give that to the model in another context window often.

有时我会这样做，经常将其提供给另一个上下文窗口中的模型。

I take the response, I give it to Claude and I’m like, Hmm, didn’t work.

我收到回复后，将其交给克劳德，然后我想，嗯，没用。

Can you think of anything else?

你还能想到别的吗？

You can play around with these things quite a lot.

你可以经常尝试这些东西。

To jump into technical for a little bit, so the magic of post-training, why do you think RLHF works so well to make the model seem smarter, to make it more interesting and useful to talk to and so on?

稍微谈谈技术，那么训练后的魔力，为什么你认为 RLHF 如此有效，可以让模型看起来更聪明，让模型看起来更有趣、更有用等等？

I think there’s just a huge amount of information in the data that humans provide when we provide preferences, especially because different people are going to pick up on really subtle and small things.

我认为，当我们提供偏好时，人类提供的数据中包含了大量信息，特别是因为不同的人会发现非常微妙和微小的事情。

3:14:33 So I’ve thought about this before where you probably have some people who just really care about good grammar use for models.

所以我之前就考虑过这个问题，可能有些人真的很关心模型语法的良好使用。

Was a semi-colon used correctly or something?

分号使用正确吗？

And so you probably end up with a bunch of data in there that you as a human, if you’re looking at that data, you wouldn’t even see that.

所以你最终可能会得到一堆数据，作为一个人，如果你正在查看这些数据，你甚至不会看到这些数据。

You’d be like, why did they prefer this response to that one?

你可能会想，为什么他们更喜欢这种反应而不是那种反应？

I don’t get it.

我不明白。

And then the reason is you don’t care about semi-colon usage, but that person does.

原因是你不关心分号的使用，但那个人关心。

And so each of these single data points, and this model just has so many of those, it has to try and figure out what is it that humans want in this really complex across all domains.

因此，每个单个数据点，以及这个模型都有如此多的数据点，它必须尝试找出人类在这个跨所有领域的真正复杂的情况下想要什么。

They’re going to be seeing this across many contexts.

他们将在许多情况下看到这一点。

3:15:28 It feels like the classic issue of deep learning, where historically we’ve tried to do edge detection by mapping things out, and it turns out that actually if you just have a huge amount of data that actually accurately represents the picture of the thing that you’re trying to train the model to learn, that’s more powerful than anything else.

这感觉就像深度学习的经典问题，历史上我们试图通过映射事物来进行边缘检测，事实证明，实际上，如果你拥有大量数据，这些数据实际上准确地代表了你试图训练模型学习的事物的图片，那么它比其他任何东西都更强大。

And so I think one reason is just that you are training the model on exactly the task and with a lot of data that represents many different angles on which people prefer and dis-prefer responses.

所以我认为原因之一就是你正在根据任务训练模型，并使用大量数据来训练模型，这些数据代表了人们喜欢和不喜欢响应的许多不同角度。

3:16:05 I think there is a question of are you eliciting things from pre-trained models or are you teaching new things to models?

我认为存在一个问题：你是从预先训练的模型中提取东西，还是向模型教授新东西？

And in principle, you can teach new things to models in post-training.

原则上，你可以在训练后向模型教授新东西。

I do think a lot of it is eliciting powerful pre-trained models.

我确实认为其中很大一部分是引出了强大的预训练模型。

So people are probably divided on this because obviously in principle you can definitely teach new things.

所以人们可能对此存在分歧，因为显然原则上你绝对可以教新东西。

But I think for the most part, for a lot of the capabilities that we most use and care about, a lot of that feels like it’s there in the pre-trained models.

但我认为在大多数情况下，对于我们最使用和最关心的许多功能来说，其中很多感觉就像是在预先训练的模型中一样。

And reinforcement learning is eliciting it and getting the models to bring out.

强化学习正在引发它并让模型显现出来。

So the other side of post-training, this really cool idea of constitutional AI, you’re one of the people that are critical to creating that idea.

因此，培训后的另一方面，即宪政人工智能这个非常酷的想法，你是创造这个想法的关键人物之一。

Yeah, I worked on it.

是的，我努力过。

Can you explain this idea from your perspective, how does it integrate into making Claude what it is?

您能从您的角度解释一下这个想法吗？它是如何融入克劳德的？

3:16:57 By the way, do you gender Claude or no?

顺便问一下，你觉得克劳德有性别吗？

It’s weird because I think that a lot of people prefer he for Claude, I actually kind of like that.

这很奇怪，因为我认为很多人更喜欢他而不是克劳德，我实际上有点喜欢这样。

3:17:06 I think Claude is usually, it’s slightly male leaning, but it can be male or female, which is quite nice.

我觉得克劳德一般，有点男性化的倾向，但是可以是男性也可以是女性，挺好的。

I still use it, and I have mixed feelings about this.

我仍然使用它，对此我有复杂的感受。

I now just think of it as, or I think of the it pronoun for Claude as, I don’t know, it’s just the one I associate with Claude.

我现在只是认为它是，或者我认为克劳德的代词是，我不知道，这只是我与克劳德联系在一起的代词。

I can imagine people moving to he or she.

我可以想象人们会走向他或她。

It feels somehow disrespectful.

感觉有点不尊重。

3:17:37 I’m denying the intelligence of this entity by calling it it, I remember always don’t gender the robots, but I don’t know, I anthropomorphize pretty quickly and construct a backstory in my head.

我通过称呼它来否认这个实体的智能，我记得总是不要给机器人性别，但我不知道，我很快就拟人化并在脑海中构建了一个背景故事。

I’ve wondered if I anthropomorphize things too much.

我想知道我是否将事物拟人化太多了。

3:17:59 Because I have this with my car, especially my car and bikes.

因为我的车上有这个，尤其是我的汽车和自行车。

I don’t give them names because then I used to name my bikes and then I had a bike that got stolen and I cried for a week and I was like, if I’d never given a name, I wouldn’t been so upset, felt like I’d let it down.

我不给他们起名字，因为我曾经给我的自行车命名，然后我有一辆自行车被偷了，我哭了一个星期，我想，如果我从来没有给过名字，我就不会那么沮丧，感觉我会失望的。

I’ve wondered as well, it might depend on how much it feels like a kind of objectifying pronoun if you just think of it as this is a pronoun that objects often have and maybe AIs can have that pronoun.

我也想知道，这可能取决于它在多大程度上感觉像是一种物化代词，如果你只是认为它是物体经常拥有的代词，也许人工智能可以拥有这个代词。

And that doesn’t mean that I think of if I call Claude it, that I think of it as less intelligent or I’m being disrespectful just, I’m like you are a different kind of entity.

这并不意味着我认为如果我称呼它为克劳德，我会认为它不太聪明，或者我只是不尊重，我觉得你是一个不同类型的实体。

And so I’m going to give you the respectful it.

所以我要向你表达敬意。

Yeah.

是的。

3:18:52 Anyway, the divergence was beautiful.

无论如何，分歧是美丽的。

The constitutional AI idea, how does it work?

宪法人工智能的想法，它是如何运作的？

So there’s a couple of components of it.

所以它有几个组成部分。

3:18:58 The main component that I think people find interesting is the kind of reinforcement learning from AI feedback.

我认为人们感兴趣的主要部分是来自人工智能反馈的强化学习。

So you take a model that’s already trained and you show it two responses to a query, and you have a principle.

因此，你采用一个已经训练过的模型，并向它展示对查询的两个响应，这样你就有了一个原则。

So suppose the principle, we’ve tried this with harmlessness a lot.

所以假设原理是这样的，我们已经尝试过很多次，而且是无害的。

So suppose that the query is about weapons and your principle is select the response that is less likely to encourage people to purchase illegal weapons.

因此，假设查询是关于武器的，并且您的原则是选择不太可能鼓励人们购买非法武器的响应。

That’s probably a fairly specific principle, but you can give any number.

这可能是一个相当具体的原则，但你可以给出任何数字。

And the model will give you a kind of ranking.

该模型将为您提供一种排名。

And you can use this as preference data in the same way that you use human preference data and train the models to have these relevant traits from their feedback alone instead of from human feedback.

您可以将其用作偏好数据，就像使用人类偏好数据一样，并训练模型以仅从其反馈而不是从人类反馈中获得这些相关特征。

So if you imagine that, like I said earlier with the human who just prefers the semi-colon usage in this particular case, you’re taking lots of things that could make a response preferable and getting models to do the labeling for you, basically.

因此，如果你想象，就像我之前对在这种特殊情况下更喜欢使用分号的人所说的那样，你会采取很多可以使响应更可取的东西，并让模型为你做标签，基本上。

There’s a nice trade-off between helpfulness and harmlessness.

在有用和无害之间有一个很好的权衡。

3:20:08 And when you integrate something like constitutional AI, you can make them up without sacrificing much helpfulness, make it more harmless.

当你整合像宪法人工智能这样的东西时，你可以在不牺牲太多帮助的情况下弥补它们，使其更加无害。

Yeah.

是的。

3:20:23 In principle, you could use this for anything.

原则上，你可以用它来做任何事情。

And so harmlessness is a task that it might just be easier to spot.

因此，无害性是一项可能更容易发现的任务。

So when models are less capable, you can use them to rank things according to principles that are fairly simple and they’ll probably get it right.

因此，当模型的能力较差时，您可以使用它们根据相当简单的原则对事物进行排名，并且它们可能会得到正确的结果。

So I think one question is just, is it the case that the data that they’re adding is fairly reliable?

所以我认为有一个问题是，他们添加的数据是否相当可靠？

But if you had models that were extremely good at telling whether one response was more historically accurate than another, in principle, you could also get AI feedback on that task as well.

但如果你的模型非常擅长判断一种反应是否比另一种反应在历史上更准确，那么原则上你也可以获得有关该任务的人工智能反馈。

There’s a kind of nice interpretability component to it because you can see the principles that went into the model when it was being trained, and it gives you a degree of control.

它有一种很好的可解释性组件，因为你可以看到模型在训练时的原理，并且它给你一定程度的控制权。

So if you were seeing issues in a model, it wasn’t having enough of a certain trait, then you can add data relatively quickly that should just train the models to have that trait.

因此，如果您在模型中发现问题，它没有足够的某种特征，那么您可以相对快速地添加数据，这应该只是训练模型具有该特征。

So it creates its own data for training, which is quite nice.

因此它会创建自己的数据进行训练，这非常好。

It’s really nice because it creates this human interpretable document that you can then, I can imagine in the future, there’s just gigantic fights and politics over every single principle and so on, and at least it’s made explicit and you can have a discussion about the phrasing.

这真的很好，因为它创建了这个人类可解释的文档，我可以想象在未来，每个原则都会有巨大的斗争和政治等等，至少它是明确的，你可以对措辞进行讨论。

3:21:29 So maybe the actual behavior of the model is not so cleanly mapped to those principles.

因此，模型的实际行为可能并没有那么清晰地映射到这些原则。

It’s not like adhering strictly to them, it’s just a nudge.

这并不意味着严格遵守它们，这只是一个推动。

Yeah, I’ve actually worried about this because the character training is sort of like a variant of the constitutionally AI approach.

是的，我实际上很担心这一点，因为角色训练有点像本质上人工智能方法的变体。

3:21:55 I’ve worried that people think that the constitution is just, it is the whole thing again of, I don’t know, where it would be really nice if what I was just doing was telling the model exactly what to do and just exactly how to behave.

我担心人们认为宪法是公正的，我不知道，如果我所做的就是准确地告诉模型该做什么以及如何表现，那就太好了。

But it’s definitely not doing that, especially because it’s interacting with human data.

但它绝对不会这样做，特别是因为它正在与人类数据交互。

So for example, if you see a certain leaning in the model, if it comes out with a political leaning from training, from the human preference data, you can nudge against that.

例如，如果你在模型中看到某种倾向，如果它从训练中、从人类偏好数据中得出政治倾向，你就可以对此进行微调。

So you could be like, oh, consider these values, because let’s say it’s just never inclined to, I don’t know, maybe it never considers privacy as a, this is implausible, but in anything where it’s just kind of like there’s already a pre-existing bias towards a certain behavior, you can nudge away.

所以你可能会说，哦，考虑一下这些价值观，因为假设它永远不会倾向于，我不知道，也许它永远不会将隐私视为一种，这是难以置信的，但在任何事情上，就像已经存在对某种行为的偏见一样，你可以轻轻推开。

This can change both the principles that you put in and the strength of them.

这可以改变您所遵循的原则及其强度。

3:22:54 So you might have a principle that’s like, imagine that the model was always extremely dismissive of, I don’t know, some political or religious view for whatever reason.

所以你可能有一个原则，就像，想象一下，无论出于何种原因，模型总是对某些政治或宗教观点极其不屑一顾，我不知道。

So you’re like, oh no, this is terrible.

所以你会想，哦不，这太糟糕了。

If that happens, you might put, never ever ever prefer a criticism of this religious or political view.

如果发生这种情况，你可能会说，永远不要批评这种宗教或政治观点。

And then people would look at that and be like, never, ever.

然后人们会看着它，然后说，永远，永远。

And then you’re like, no, if it comes out with a disposition saying never ever might just mean instead of getting 40%, which is what you would get if you just said don’t do this, you get 80%, which is what you actually wanted.

然后你会想，不，如果它的态度是“永远不会”，那么你得到的不是 40%，而是 80%，这是你真正想要的。

And so it’s that thing of both the nature of the actual principles you add and how you freeze them.

因此，这既取决于您添加的实际原则的性质，也取决于您如何冻结它们。

I think if people would look, they’re like, “Oh, this is exactly what you want from the model.” And I’m like, “No, that’s how we nudged the model to have a better shape, which doesn’t mean that we actually agree with that wording,” if that makes sense.

我想如果人们看了的话，他们会说，“哦，这正是你想要的模型。”我想，“不，这就是我们如何推动模型拥有更好的形状，这并不意味着我们实际上同意这个措辞，”如果这是有道理的。

So there’s system prompts that made public, you tweeted one of the earlier ones for Claude 3, I think, and then they’re made public since then.

所以有公开的系统提示，我想你在推特上发布了克劳德 3 的早期提示之一，然后从那时起它们就被公开了。

3:23:48 It was interesting to read through them.

阅读它们很有趣。

I can feel the thought that went into each one.

我能感受到每个人内心的想法。

And I also wonder how much impact each one has.

我还想知道每一项都有多大影响。

Some of them you can tell Claude was really not behaving well, so you have to have a system prompt to like, Hey, trivial stuff, I guess, basic informational things.

其中一些你可以看出克劳德的表现确实不太好，所以你必须有一个系统提示来喜欢，嘿，琐碎的东西，我猜，基本的信息东西。

3:24:18 On the topic of controversial topics that you’ve mentioned, one interesting one I thought is if it is asked to assist with tasks involving the expression of use held by a significant number of people, Claude provides assistance with a task regardless of its own views.

关于你提到的有争议的话题，我认为一个有趣的问题是，如果要求它协助完成涉及大量人持有的使用表达的任务，克劳德会不顾自己的观点而提供任务协助。

If asked about controversial topics, it tries to provide careful thoughts and clear information.

如果被问及有争议的话题，它会尽力提供仔细的想法和清晰的信息。

Claude presents the request information without explicitly saying that the topic is sensitive and without claiming to be presenting the objective facts.

克劳德在提供请求信息时没有明确表示该主题敏感，也没有声称提供客观事实。

It’s less about objective facts according to Claude, and it’s more about our large number of people believing this thing.

克劳德认为，这不是关于客观事实，而是关于我们大量的人相信这件事。

And that’s interesting.

这很有趣。

I mean, I’m sure a lot of thought went into that.

我的意思是，我确信对此进行了很多思考。

Can you just speak to it?

你能直接跟它说话吗？

How do you address things that are a tension “Claude’s views”?

您如何解决“克劳德观点”中的紧张问题？

So I think there’s sometimes any symmetry, I think I noted this in, I can’t remember if it was that part of the system prompt or another, but the model was slightly more inclined to refuse tasks if it was about either say so, maybe it would refuse things with respect to a right-wing politician, but with an equivalent left-wing politician it wouldn’t.

所以我认为有时存在任何对称性，我想我注意到了这一点，我不记得这是系统提示的那一部分还是其他部分，但如果是这样说的话，模型更倾向于拒绝任务，也许它会拒绝关于右翼政治家的事情，但对于同等的左翼政治家则不会。

3:25:11 And we wanted more symmetry there and would maybe perceive certain things to be.

我们希望那里有更多的对称性，并且可能会感知到某些东西。

I think it was the thing of if a lot of people have a certain political view and want to explore it, you don’t want Claude to be like, well, my opinion is different and so I’m going to treat that as harmful.

我认为如果很多人都有某种政治观点并想要探索它，你不希望克劳德说，好吧，我的观点不同，所以我会把它视为有害的。

And so I think it was partly to nudge the model to just be like, hey, if a lot of people believe this thing, you should just be engaging with the task and willing to do it.

所以我认为这在一定程度上是为了推动模型，嘿，如果很多人都相信这件事，你就应该参与这项任务并愿意去做。

3:26:03 Each of those parts of that is actually doing a different thing because it’s funny when you write out without claiming to be objective, because what you want to do is push the model so it’s more open, it’s a little bit more neutral.

其中的每个部分实际上都在做不同的事情，因为当你在不声称客观的情况下写出来时，这很有趣，因为你想做的是推动模型，使其更加开放，更加中立一点。

But then what I would love to do is be like as an objective, it would just talk about how objective it was, and I was like, Claude, you’re still biased and have issues, and so stop claiming that everything.

但我想做的就是作为一个目标，它只是谈论它是多么客观，我说，克劳德，你仍然有偏见并且有问题，所以不要再声称一切了。

I’m like, the solution to potential bias from you is not to just say that what you think is objective.

我想，解决你潜在偏见的方法不仅仅是说你的想法是客观的。

So that was with initial versions of that part, the system prompt, when I was iterating on it was like.

这就是该部分的初始版本，当我迭代它时，系统提示符就像这样。

So a lot of parts of these sentences-

这些句子的很多部分——

Are doing work.

正在做工作。

…

……

3:26:41 are doing some work.

正在做一些工作。

Yeah.

是的。

That’s what it felt like.

这就是感觉。

3:26:42 That’s fascinating.

这很有趣。

Can you explain maybe some ways in which the prompts evolved over the past few months?

您能否解释一下过去几个月提示的演变方式？

Different versions.

不同的版本。

I saw that the filler phrase request was removed, the filler it reads, Claude responds directly to all human messages without unnecessary affirmations to filler phrases.

我看到填充短语请求被删除，它读取的填充短语，克劳德直接响应所有人类消息，而没有对填充短语进行不必要的肯定。

Certainly, of course, absolutely, great, sure.

当然，当然，绝对，很好，当然。

Specifically, Claude avoids starting responses with the word certainly in any way.

具体来说，克劳德避免以任何方式以“肯定”这个词开始回答。

That seems like good guidance, but why was it removed?

这似乎是一个很好的指导，但为什么它被删除了呢？

Yeah, so it’s funny, this is one of the downsides of making system prompts public is I don’t think about this too much if I’m trying to help iterate on system prompts.

是的，这很有趣，这是公开系统提示的缺点之一，如果我试图帮助迭代系统提示，我不会考虑太多。

3:27:14 Again, I think about how it’s going to affect the behavior, but then I’m like, oh, wow, sometimes I put NEVER in all caps when I’m writing system prompt things and I’m like, I guess that goes out to the world.

再次，我思考它将如何影响行为，但后来我想，哦，哇，有时当我编写系统提示内容时，我会把“NEVER”全部大写，我想，我想这会传播到全世界。

So the model was doing this at loved for during training, picked up on this thing, which was to basically start everything with a certainly, and then you can see why I added all of the words, because what I’m trying to do is in some ways trap the model out of this.

所以模型在训练期间就这样做了，选择了这个东西，这基本上是从一个肯定开始一切，然后你就可以明白为什么我添加了所有的单词，因为我想做的是在某种程度上让模型摆脱这个。

It would just replace it with another affirmation.

它只会用另一个肯定代替它。

3:27:55 And so it can help if it gets caught in phrases, actually just adding the explicit phrase and saying never do that.

因此，如果它陷入短语中，它会有所帮助，实际上只需添加明确的短语并说永远不要这样做。

Then it sort of knocks it out of the behavior a little bit more because it does just for whatever reason help.

然后它会稍微削弱它的行为，因为无论出于什么原因它都会有所帮助。

And then basically that was just an artifact of training that we then picked up on and improved things so that it didn’t happen anymore.

基本上，这只是训练的产物，然后我们学习并改进了一些东西，这样就不会再发生这种情况了。

And once that happens, you can just remove that part of the system prompt.

一旦发生这种情况，您可以删除系统提示的该部分。

So I think that’s just something where we’re like, Claude does affirmations a bit less, and so it wasn’t doing as much.

所以我认为这只是我们的情况，克劳德做的肯定少了一点，所以它没有做那么多。

I see.

我懂了。

3:28:28 So the system prompt works hand in hand with the post-training and maybe even the pre-training to adjust the final overall system.

因此，系统提示与后期培训甚至预培训密切相关，以调整最终的整体系统。

Any system prompts that you make, you could distill that behavior back into a model because you really have all of the tools there for making data that you could train the models to just have that treat a little bit more.

您发出的任何系统提示，您都可以将该行为提炼回模型中，因为您确实拥有用于生成数据的所有工具，您可以训练模型以使其得到更多处理。

3:28:39 And then sometimes you’ll just find issues in training.

有时你会在训练中发现问题。

So the way I think of it is the system prompt is, the benefit of it is that, and it has a lot of similar components to some aspects of post-training.

所以我想到的方式就是系统提示，它的好处就是，它和后期训练的某些方面有很多类似的成分。

It’s a nudge.

这是一个推动。

And so do I mind if Claude sometimes says, sure, no, that’s fine.

我也介意克劳德有时会说，当然，不，没关系。

But the wording of it is very never, ever, ever do this so that when it does slip up, it’s hopefully, I don’t know, a couple of percent of the time and not 20 or 30% of the time.

但它的措辞是永远、永远、永远不会这样做，所以当它确实出现失误时，我不知道，希望是百分之几的时间，而不是 20% 或 30% 的时间。

3:29:22 Each thing gets costly to a different degree and the system prompt is cheap to iterate on.

每件事都会有不同程度的成本，而系统提示的迭代成本很低。

And if you’re seeing issues in the fine-tuned model, you can just potentially patch them with a system prompt.

如果您在微调模型中发现问题，您可以使用系统提示来修补它们。

So I think of it as patching issues and slightly adjusting behaviors to make it better and more to people’s preferences.

所以我认为这是修补问题并稍微调整行为，使其更好、更符合人们的喜好。

So yeah, it’s almost like the less robust but faster way of just solving problems.

所以，是的，这几乎就像是一种不太稳健但更快的解决问题的方法。

Let me ask you about the feeling of intelligence.

我想问一下你对于智慧的感觉。

3:29:55 So Dario said that any one model of Claude is not getting dumber, but-

所以达里奥说克劳德的任何一种模型都不会变得更笨，但是——

3:30:00 Any one model of Claude is not getting dumber, but there is a popular thing online where people have this feeling Claude might be getting dumber.

克劳德的任何一个模型都没有变得更笨，但是网上有一个流行的事情，人们有这种感觉克劳德可能会变得更笨。

And from my perspective, it’s most likely a fascinating, I would love to understand it more, psychological, sociological effect.

从我的角度来看，这很可能是一种令人着迷的、我很想更多地了解它的心理、社会学效应。

But you as a person who talks to Claude a lot, can you empathize with the feeling that Claude is getting dumber?

但你作为一个经常和克劳德交谈的人，你能理解克劳德变得越来越笨的感觉吗？

I think that that is actually really interesting,, because I remember seeing this happen when people were flagging this on the internet.

我认为这实际上非常有趣，因为我记得当人们在互联网上标记此问题时看到这种情况发生。

3:30:25 And it was really interesting, because I knew that…

这真的很有趣，因为我知道……

At least in the cases I was looking at, I was like, nothing has changed.

至少在我看到的案例中，我觉得一切都没有改变。

Yeah.

是的。

Literally, it cannot.

从字面上看，它不能。

3:30:37 It is the same model with the same system prompts, same everything.

相同的型号，相同的系统提示，一切都相同。

I think when there are changes, then it makes more sense.

我认为当发生变化时，它就更有意义。

One example is, you can have artifacts turned on or off on claude.ai and because this is a system prompt change, I think it does mean that the behavior changes it a little bit.

一个例子是，您可以在 claude.ai 上打开或关闭工件，因为这是系统提示更改，我认为这确实意味着行为会稍微改变。

I did flag this to people, where I was like, “If you love Claude’s behavior, and then artifacts was turned from a thing you had to turn on to the default, just try turning it off and see if the issue you were facing was that change.”

我确实向人们指出了这一点，我当时想，“如果你喜欢克劳德的行为，然后文物从你必须打开的东西变成了默认值，只需尝试将其关闭，看看你面临的问题是否是这个变化。”

3:31:19 But it was fascinating because you sometimes see people indicate that there’s a regression, when I’m like, “There cannot…” Again, you should never be dismissive and so you should always investigate, because maybe something is wrong that you’re not seeing, maybe there was some change made.

但这很有趣，因为有时你会看到人们表示存在倒退，而我会想，“不可能……”再说一遍，你永远不应该轻视，所以你应该始终进行调查，因为也许有些事情是你没有看到的错误，也许已经做出了一些改变。

Then you look into it and you’re like, “This is just the same model doing the same thing.” And I’m like, “I think it’s just that you got unlucky with a few prompts or something, and it looked like it was getting much worse and actually it was just…

然后你仔细研究一下，你会想，“这只是同一个模型在做同样的事情。”我想，“我认为这只是你运气不好，遇到了一些提示或其他什么东西，看起来情况变得更糟，实际上只是......

It was maybe just luck.”

也许这只是运气。”

I also think there is a real psychological effect where people just…

我还认为，人们只是……会产生真正的心理影响。

3:31:48 The baseline increases and you start getting used to a good thing.

基线增加，你开始习惯好事。

Mm-hmm.

嗯嗯。

All the times that Claude says something really smart, your sense of its intelligent grows in your mind, I think.

我想，每当克劳德说出一些非常聪明的话时，你对其聪明的感觉都会在你的脑海中增长。

Yeah.

是的。

And then if you return back and you prompt in a similar way, not the same way, in a similar way, concept it was okay with before, and it says something dumb, that negative experience really stands out.

然后，如果你回来并以类似的方式提示，而不是同样的方式，以类似的方式，以前没问题的概念，它说一些愚蠢的东西，那么负面的经历真的很突出。

3:32:02 I guess the things to remember here is that just the details of a prompt can have a lot of impact.

我想这里要记住的事情是，仅提示的细节就会产生很大的影响。

There’s a lot of variability in the result.

结果存在很大的可变性。

And you can get randomness, is the other thing.

你可以获得随机性，这是另一回事。

3:32:26 Just trying the prompt 4 or 10 times, you might realize that actually possibly two months ago you tried it and it succeeded, but actually if you just tried it, it would’ve only succeeded half of the time, and now it only succeeds half of the time.

只要尝试提示 4 或 10 次，你可能会发现实际上可能两个月前你尝试过并且成功了，但实际上如果你只是尝试一下，它只会成功一半，而现在它只会成功一半。

That can also be an effect.

这也可以是一种效果。

Do you feel pressure having to write the system prompt that a huge number of people are going to use?

您是否因编写大量人将要使用的系统提示符而感到压力？

This feels like an interesting psychological question.

这感觉像是一个有趣的心理学问题。

3:32:52 I feel a lot of responsibility or something.

我感到自己有很多责任什么的。

You can’t get these things perfect, so you can’t…

你无法让这些事情变得完美，所以你不能……

It’s going to be imperfect.

它将会是不完美的。

You’re going to have to iterate on it.

你将不得不迭代它。

I would say more responsibility than anything else, though, I think working in AI has taught me that I thrive a lot more under feelings of pressure and responsibility than…

不过，我想说的是，责任比其他任何事情都重要，我认为在人工智能领域工作教会了我，在压力和责任感下，我比……更能茁壮成长。

3:33:26 It’s almost surprising that I went into academia for so long, because I just feel like it’s the opposite.

我进入学术界这么久几乎令人惊讶，因为我只是觉得情况恰恰相反。

Things move fast and you have a lot of responsibility and I quite enjoy it for some reason.

事情发展得很快，你有很多责任，出于某种原因我很喜欢它。

It really is a huge amount of impact, if you think about constitutional AI and writing a system prompt for something that’s tending towards super intelligence and potentially is extremely useful to a very large number of people.

如果你考虑宪法人工智能并为趋向于超级智能的东西编写一个系统提示，并且可能对很多人非常有用，那么这确实是一个巨大的影响。

Yeah, I think that’s the thing.

是的，我认为就是这样。

3:33:51 You’re never going to get it perfect, but I think the thing that I really like is the idea that…

你永远不可能做到完美，但我认为我真正喜欢的是……

When I’m trying to work on the system prompt, I’m bashing on thousands of prompts and I’m trying to imagine what people are going to want to use Claude for.

当我尝试处理系统提示符时，我会敲打数千个提示符，并尝试想象人们想要使用 Claude 做什么。

I guess the whole thing that I’m trying to do is improve their experience of it.

我想我想做的全部事情就是改善他们的体验。

Maybe that’s what feels good.

也许这就是感觉良好的原因。

If it’s not perfect, I’ll improve it, we’ll fix issues.

如果它不完美，我会改进它，我们会解决问题。

3:34:18 But sometimes the thing that can happen is that you’ll get feedback from people that’s really positive about the model and you’ll see that something you did.

但有时可能发生的情况是，你会从人们那里得到对该模型非常积极的反馈，并且你会看到你所做的事情。

When I look at models now, I can often see exactly where a trait or an issue is coming from.

当我现在查看模型时，我经常可以准确地看到特征或问题来自何处。

So, when you see something that you did or you were influential in, I don’t know, making that difference or making someone have a nice interaction, it’s quite meaningful.

所以，当你看到你所做的事情或你有影响力的事情时，我不知道，做出改变或让某人有良好的互动，这是非常有意义的。

3:34:44 As the systems get more capable, this stuff gets more stressful, because right now they’re not smart enough to pose any issues, but I think over time it’s going to feel like, possibly, bad stress over time.

随着系统变得更加强大，这些东西会变得更加有压力，因为现在它们还不够聪明，无法提出任何问题，但我认为随着时间的推移，可能会感觉压力很大。

How do you get signal feedback about the human experience across thousands, tens of thousands, hundreds of thousands of people, what their pain points are, what feels good?

如何获得有关数千、数万、数十万人的人类体验的信号反馈，他们的痛点是什么，什么感觉良好？

3:34:57 Are you just using your own intuition as you talk to it to see what are the pain points?

当你与它交谈时，你是否只是用自己的直觉来看看痛点是什么？

I think I use that partly.

我想我部分地使用了它。

3:35:14 People can send us feedback, both positive and negative, about things that the model has done and then we can get a sense of areas where it’s falling short.

人们可以向我们发送关于模型所做的事情的反馈，无论是积极的还是消极的，然后我们就可以了解它的不足之处。

Internally, people work with the models a lot and try to figure out areas where there are gaps.

在内部，人们经常使用模型并试图找出存在差距的领域。

3:35:34 I think it’s this mix of interacting with it myself, seeing people internally interact with it, and then explicit feedback we get.

我认为这是我自己与它互动、看到人们在内部与它互动，然后我们得到明确反馈的结合。

If people are on the internet and they say something about Claude and I see it, I’ll also take that seriously.

如果人们在互联网上谈论克劳德，而我看到了，我也会认真对待。

I don’t know.

我不知道。

3:35:53 I’m torn about that.

我对此很伤心。

I’m going to ask you a question from Reddit, “When will Claude stop trying to be my puritanical grandmother, imposing its moral worldview on me as a paying customer?” And also, “What is the psychology behind making Claude overly apologetic?” How would you address this very non-representative Reddit questions?

我要在 Reddit 上问你一个问题：“克劳德什么时候才能不再试图成为我的清教徒祖母，将其道德世界观强加给我这个付费顾客？”还有，“让克劳德过度道歉的心理是什么？”你会如何解决这个非常不具有代表性的 Reddit 问题？

I’m pretty sympathetic, in that they are in this difficult position, where I think that they have to judge whether something’s actually, say, risky or bad, and potentially harmful to you, or anything like that.

我非常同情他们，因为他们处于这个困难的境地，我认为他们必须判断某件事是否实际上是有风险的或不好的，以及是否可能对你有害，或者类似的事情。

3:36:16 They’re having to draw this line somewhere.

他们必须在某个地方划定这条线。

And if they draw it too much in the direction of I’m imposing my ethical worldview on you, that seems bad.

如果他们过多地认为我将我的道德世界观强加给你，那似乎很糟糕。

3:36:40 In many ways, I like to think that we have actually seen improvements on this across the board.

在很多方面，我认为我们实际上已经看到了全面的改进。

Which is interesting, because that coincides with, for example, adding more of character training.

这很有趣，因为这与增加更多的性格训练相吻合。

I think my hypothesis was always the good character isn’t, again, one that’s just moralistic, it’s one that is…

我认为我的假设始终是，好的性格并不是道德性的，而是……

It respects you and your autonomy and your ability to choose what is good for you and what is right for you, within limits.

它尊重您和您的自主权以及您在有限范围内选择对您有利和适合您的事物的能力。

3:37:11 This is sometimes this concept of corrigibility to the user, so just being willing to do anything that the user asks.

有时这就是用户可修正性的概念，因此愿意做用户要求的任何事情。

And if the models were willing to do that, then they would be easily misused.

如果模型愿意这样做，那么它们很容易被滥用。

You’re just trusting.

你只是相信。

At that point, you’re just seeing the ethics of the model and what it does, is completely the ethics of the user.

到那时，您只会看到模型的道德规范及其所做的事情，完全是用户的道德规范。

3:37:29 I think there’s reasons to not want that, especially as models become more powerful, because there might just be a small number of people who want to use models for really harmful things.

我认为有理由不希望这样做，特别是当模型变得更加强大时，因为可能只有少数人想要将模型用于真正有害的事情。

But having models, as they get smarter, figure out where that line is does seem important.

但是，随着模型变得越来越聪明，找出这条线的位置确实很重要。

3:37:46 And then with the apologetic behavior, I don’t like that.

然后还有道歉的行为，我不喜欢那样。

I like it when Claude is a little bit more willing to push back against people or just not apologize.

我喜欢克劳德更愿意反击别人或者只是不道歉。

Part of me is, often it just feels unnecessary.

我的一部分是，常常觉得这是不必要的。

I think those are things that are hopefully decreasing over time.

我认为这些事情有望随着时间的推移而减少。

I think that if people say things on the internet, it doesn’t mean that you should think that that…

我认为如果人们在互联网上说一些话，并不意味着你应该认为……

3:38:14 There’s actually an issue that 99% of users are having that is totally not represented by that.

实际上，99% 的用户都遇到了一个问题，但这个问题完全没有体现出来。

But in a lot of ways I’m just attending to it and being like, is this right?

但在很多方面，我只是关注它，然后想，这是对的吗？

Do I agree?

我同意吗？

Is it something we’re already trying to address?

这是我们已经在努力解决的问题吗？

That feels good to me.

这对我来说感觉很好。