Distinct AI Models Seem To Converge On How They Encode Reality
Do all AI models represent “cat” in the same way?
所有人工智能模型都以同样的方式代表“猫”吗？
Read a story about dogs, and you may remember it the next time you see one bounding through a park.
读一个关于狗的故事，下次当你看到一只狗在公园里跳跃时，你可能会记住它。
That’s only possible because you have a unified concept of “dog” that isn’t tied to words or images alone.
这是可能的，因为你有一个统一的“狗”概念，而不仅仅与文字或图像联系在一起。
Bulldog or border collie, barking or getting its belly rubbed, a dog can be many things while still remaining a dog.
斗牛犬或边境牧羊犬，吠叫或被抚摸，一只狗可以做很多事情，同时仍然是一只狗。
Artificial intelligence systems aren’t always so lucky.
人工智能系统并不总是那么幸运。
These systems learn by ingesting vast troves of data in a process called training.
这些系统通过在称为训练的过程中摄取大量数据来进行学习。
Often, that data is all of the same type — text for language models, images for computer vision systems, and more exotic kinds of data for systems designed to predict the odor of molecules or the structure of proteins .
通常，这些数据都是同一类型——用于语言模型的文本、用于计算机视觉系统的图像，以及用于预测分子气味或蛋白质结构的系统的更奇特的数据类型。
So to what extent do language models and vision models have a shared understanding of dogs?
那么语言模型和视觉模型对狗的理解在多大程度上有共同点呢？
As AI models grow more powerful, they appear to be converging on how they internally represent reality.
随着人工智能模型变得越来越强大，它们似乎正在向内部表示现实的方式趋同。
Researchers investigate such questions by peering inside AI systems and studying how they represent scenes and sentences.
研究人员通过深入人工智能系统内部并研究它们如何表示场景和句子来研究这些问题。
A growing body of research has found that different AI models can develop similar representations, even if they’re trained using different datasets or entirely different data types.
越来越多的研究发现，不同的人工智能模型可以开发相似的表示，即使它们是使用不同的数据集或完全不同的数据类型进行训练的。
What’s more, a few studies have suggested that those representations are growing more similar as models grow more capable.
更重要的是，一些研究表明，随着模型能力的增强，这些表征也变得越来越相似。
In a 2024 paper , four AI researchers at the Massachusetts Institute of Technology argued that these hints of convergence are no fluke.
在 2024 年的一篇论文中，麻省理工学院的四名人工智能研究人员认为，这些融合的迹象并非侥幸。
Their idea, dubbed the Platonic representation hypothesis, has inspired a lively debate among researchers and a slew of follow-up work .
他们的想法被称为柏拉图表示假说，激发了研究人员之间的激烈辩论和大量后续工作。
The team’s hypothesis gets its name from a 2,400-year-old allegory by the Greek philosopher Plato.
该团队的假设得名于希腊哲学家柏拉图的一个 2400 年前的寓言。
In it, prisoners trapped inside a cave perceive the world only through shadows cast by outside objects.
在其中，被困在山洞里的囚犯只能通过外部物体投射的阴影来感知世界。
Plato maintained that we’re all like those unfortunate prisoners.
柏拉图坚持认为，我们都像那些不幸的囚犯。
The objects we encounter in everyday life, in his view, are pale shadows of ideal “forms” that reside in some transcendent realm beyond the reach of the senses.
在他看来，我们在日常生活中遇到的物体是理想“形式”的苍白阴影，存在于感官无法触及的超然境界中。
An allegory by the Greek philosopher Plato inspired a recent paper about AI systems.
希腊哲学家柏拉图的寓言启发了最近一篇关于人工智能系统的论文。
Silanion at the Capitoline Museums
卡比托利欧博物馆的 Silanion
The Platonic representation hypothesis is less abstract.
柏拉图式的表征假设不那么抽象。
In this version of the metaphor, what’s outside the cave is the real world, and it casts machine-readable shadows in the form of streams of data.
在这个版本的比喻中，洞穴外面就是现实世界，它以数据流的形式投射出机器可读的阴影。
AI models are the prisoners.
人工智能模型是囚犯。
The MIT team’s claim is that very different models, exposed only to the data streams, are beginning to converge on a shared “Platonic representation” of the world behind the data.
麻省理工学院团队声称，仅暴露于数据流的非常不同的模型开始收敛于数据背后世界的共享“柏拉图式表示”。
“Why do the language model and the vision model align?
“为什么语言模型和视觉模型是一致的？
Because they’re both shadows of the same world,” said Phillip Isola , the senior author of the paper.
因为它们都是同一个世界的影子，”该论文的资深作者菲利普·伊索拉（Phillip Isola）说。
Not everyone is convinced.
并非所有人都相信。
One of the main points of contention involves which representations to focus on.
争论的要点之一涉及应该关注哪些表现形式。
You can’t inspect a language model’s internal representation of every conceivable sentence, or a vision model’s representation of every image.
您无法检查语言模型对每个可想象的句子的内部表示，或视觉模型对每个图像的表示。
So how do you decide which ones are, well, representative?
那么你如何决定哪些是具有代表性的呢？
Where do you look for the representations, and how do you compare them across very different models?
您在哪里寻找这些表示，以及如何在非常不同的模型中比较它们？
It’s unlikely that researchers will reach a consensus on the Platonic representation hypothesis anytime soon, but that doesn’t bother Isola.
研究人员不太可能很快就柏拉图表示假说达成共识，但这并不困扰伊索拉。
“Half the community says this is obvious, and the other half says this is obviously wrong,” he said.
“一半的社区认为这是显而易见的，而另一半则认为这显然是错误的，”他说。
“We were happy with that response.”
“我们对这个回应感到满意。”
If AI researchers don’t agree on Plato, they might find more common ground with his predecessor Pythagoras, whose philosophy supposedly started from the premise “All is number.” That’s an apt description of the neural networks that power AI models.
如果人工智能研究人员不同意柏拉图的观点，他们可能会与他的前任毕达哥拉斯找到更多共同点，据说毕达哥拉斯的哲学始于“一切都是数字”的前提。这是对驱动人工智能模型的神经网络的恰当描述。
Their representations of words or pictures are just long lists of numbers, each indicating the degree of activation of a specific artificial neuron.
它们对文字或图片的表示只是一长串数字，每个数字表示特定人工神经元的激活程度。
To simplify the math, researchers typically focus on a single layer of a neural network in isolation, which is akin to taking a snapshot of brain activity in a specific region at a specific moment in time.
为了简化数学计算，研究人员通常专注于孤立的神经网络的单层，这类似于在特定时刻拍摄特定区域的大脑活动快照。
They write down the neuron activations in this layer as a geometric object called a vector — an arrow that points in a particular direction in an abstract space.
他们将这一层中的神经元激活写成一个称为向量的几何对象——一个指向抽象空间中特定方向的箭头。
Modern AI models have many thousands of neurons in each layer, so their representations are high-dimensional vectors that are impossible to visualize directly.
现代人工智能模型每层都有数千个神经元，因此它们的表示是高维向量，无法直接可视化。
But vectors make it easy to compare a network’s representations: Two representations are similar if the corresponding vectors point in similar directions.
但是向量可以很容易地比较网络的表示：如果相应的向量指向相似的方向，则两个表示是相似的。
Within a single AI model, similar inputs tend to have similar representations.
在单个人工智能模型中，相似的输入往往具有相似的表示。
In a language model, for instance, the vector representing the word “dog” will be relatively close to vectors representing “pet,” “bark,” and “furry,” and farther from “Platonic” and “molasses.” It’s a precise mathematical realization of an idea memorably expressed more than 60 years ago by the British linguist John Rupert Firth: “You shall know a word by the company it keeps.”
例如，在语言模型中，表示“狗”一词的向量将相对接近表示“宠物”、“树皮”和“毛茸茸”的向量，而远离“柏拉图式”和“糖蜜”。这是英国语言学家约翰·鲁珀特·费斯 (John Rupert Firth) 在 60 多年前提出的一个令人难忘的想法的精确数学实现：“你应该通过它的同伴来认识一个单词。”
What about representations in different models?
不同模型中的表示又如何呢？
It doesn’t make sense to directly compare activation vectors from separate networks, but researchers have devised indirect ways to assess representational similarity.
直接比较来自不同网络的激活向量是没有意义的，但研究人员设计了间接方法来评估表征相似性。
One popular approach is to embrace the lesson of Firth’s pithy quote and measure whether two models’ representations of an input keep the same company.
一种流行的方法是接受弗斯精辟引述的教训，并衡量两个模型对输入的表示是否保持一致。
Imagine that you want to compare how two language models represent words for animals.
想象一下，您想要比较两种语言模型如何表示动物单词。
First, you’ll compile a list of words — dog, cat, wolf, jellyfish, and so on.
首先，您将编制一个单词列表——狗、猫、狼、水母等等。
You’ll then feed these words into both networks and record their representations of each word.
然后，您将这些单词输入两个网络并记录它们对每个单词的表示。
In each network, the representations will form a cluster of vectors.
在每个网络中，表示将形成向量簇。
You can then ask: How similar are the overall shapes of the two clusters?
然后你可以问：两个星团的整体形状有多相似？
“It can kind of be described as measuring the similarity of similarities,” said Ilia Sucholutsky , an AI researcher at New York University.
纽约大学人工智能研究员 Ilia Sucholutsky 表示：“它可以被描述为衡量相似性的相似性。”
In this simple example, you’d expect some similarity between the two models — the “cat” vector would probably be close to the “dog” vector in both networks, for instance, and the “jellyfish” vector would point in a different direction.
在这个简单的示例中，您会期望两个模型之间存在一些相似性 - 例如，“猫”向量可能接近两个网络中的“狗”向量，而“水母”向量将指向不同的方向。
But the two clusters probably won’t look exactly the same.
但这两个星团看起来可能并不完全相同。
Is “dog” more like “cat” than “wolf,” or vice versa?
“狗”更像“猫”而不是“狼”，还是反之亦然？
If your models were trained on different datasets, or built on different network architectures, they might not agree.
如果您的模型是在不同的数据集上进行训练的，或者是在不同的网络架构上构建的，那么它们可能会不同意。
Researchers began to explore representational similarity among AI models with this approach in the mid-2010s and found that different models’ representations of the same concepts were often similar, though far from identical.
研究人员在 2010 年代中期开始使用这种方法探索人工智能模型之间的表征相似性，发现不同模型对相同概念的表征往往相似，但远非完全相同。
Intriguingly, a few studies found that more powerful models seemed to have more similarities in their representations than weaker ones.
有趣的是，一些研究发现，更强大的模型在表示方面似乎比较弱的模型有更多相似之处。
One 2021 paper dubbed this the “ Anna Karenina scenario ,” a nod to the opening line of the classic Tolstoy novel .
2021 年的一篇论文将其称为“安娜·卡列尼娜情景”，这是对托尔斯泰经典小说开场白的致敬。
Perhaps successful AI models are all alike, and every unsuccessful model is unsuccessful in its own way.
也许成功的人工智能模型都是相似的，而每一个不成功的模型都有自己的不成功。
That paper, like much of the early work on representational similarity, focused only on computer vision, which was then the most popular branch of AI research.
这篇论文与许多关于表征相似性的早期研究一样，只关注计算机视觉，而计算机视觉是当时人工智能研究中最受欢迎的分支。
The advent of powerful language models was about to change that.
强大的语言模型的出现即将改变这一现状。
For Isola, it was also an opportunity to see just how far representational similarity could go.
对于伊索拉来说，这也是一个机会来了解代表性相似性到底能走多远。
The story of the Platonic representation hypothesis paper began in early 2023, a turbulent time for AI researchers.
柏拉图式表征假说论文的故事始于 2023 年初，对于人工智能研究人员来说，这是一个动荡的时期。
ChatGPT had been released a few months before, and it was increasingly clear that simply scaling up AI models — training larger neural networks on more data — made them better at many different tasks.
ChatGPT 已于几个月前发布，越来越明显的是，简单地扩展人工智能模型（利用更多数据训练更大的神经网络）可以让它们在许多不同的任务上表现得更好。
But it was unclear why.
但尚不清楚原因。
“Everyone in AI research was going through an existential life crisis,” said Minyoung Huh , an OpenAI researcher who was a graduate student in Isola’s lab at the time.
“人工智能研究领域的每个人都在经历生存危机，”OpenAI 研究员 Minyoung Huh 说，他当时是 Isola 实验室的研究生。
He began meeting regularly with Isola and their colleagues Brian Cheung and Tongzhou Wang to discuss how scaling might affect internal representations.
他开始定期与 Isola 及其同事 Brian Cheung 和 Tongzhou Wang 会面，讨论规模扩大可能如何影响内部代表。
Clockwise from top right: Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola proposed that different AI models are converging toward a shared “Platonic representation” of the world behind their training data.
从右上角顺时针方向：Minyoung Huh、Brian Cheung、Tongzhou Wang 和 Phillip Isola 提出，不同的人工智能模型正在朝着训练数据背后的世界共享“柏拉图式表示”。
From top right: Anna Decker; @by.h_official; Jiaxi Chen; Kris Brewer
右上起：安娜·德克尔； @by.h_official；陈嘉熙;克里斯·布鲁尔
Imagine a case where multiple models are trained on the same data, and the stronger models learn more similar representations.
想象一下这样的情况：多个模型在相同的数据上进行训练，并且更强的模型学习更多相似的表示。
This isn’t necessarily because these models are creating a more accurate likeness of the world.
这并不一定是因为这些模型正在创造一个更准确的世界相似性。
They could just be better at grasping quirks of the training dataset.
他们可以更好地掌握训练数据集的怪癖。
Now consider models trained on different datasets.
现在考虑在不同数据集上训练的模型。
If their representations also converge, that would be more compelling evidence that models are getting better at grasping shared features of the world behind the data.
如果它们的表示也收敛，这将是更有说服力的证据，表明模型正在更好地掌握数据背后世界的共同特征。
Convergence between models that learned from entirely different data types, such as language and vision models, would provide even stronger evidence.
从完全不同的数据类型（例如语言和视觉模型）学习的模型之间的收敛将提供更有力的证据。
A year after their initial conversations, Isola and his colleagues decided to write a paper reviewing the evidence for convergent representations and presenting an argument for the Platonic representation hypothesis.
初次对话一年后，伊索拉和他的同事决定写一篇论文，回顾收敛表征的证据，并提出柏拉图表征假说的论证。
By then, other researchers had started studying similarities between vision and language model representations.
那时，其他研究人员已经开始研究视觉和语言模型表示之间的相似性。
Huh conducted his own experiment, in which he tested a set of five vision models and 11 language models of varying sizes on a dataset of captioned pictures from Wikipedia.
Huh 进行了自己的实验，在维基百科的带字幕图片数据集上测试了一组 5 个视觉模型和 11 个不同大小的语言模型。
He would feed the pictures into the vision models and the captions into the language models, and then compare clusters of vectors in the two types.
他将图片输入视觉模型，将字幕输入语言模型，然后比较这两种类型的向量簇。
He observed a steady increase in representational similarity as models became more powerful.
他观察到，随着模型变得更加强大，表征相似性稳步增加。
It was exactly what the Platonic representation hypothesis predicted.
这正是柏拉图表征假说所预言的。
Of course, it’s never so simple.
当然，事情从来没有这么简单。
Measurements of representational similarity invariably involve a host of experimental choices that can affect the outcome.
表征相似性的测量总是涉及大量可能影响结果的实验选择。
Which layers do you look at in each network?
您在每个网络中查看哪些层？
Once you have a cluster of vectors from each model, which of the many mathematical methods do you use to compare them?
一旦获得了每个模型的向量簇，您将使用多种数学方法中的哪一种来比较它们？
And which representations do you measure in the first place?
您首先测量哪些表征？
“If you only test one dataset, you don’t necessarily know how [the result] generalizes,” said Christopher Wolfram, a researcher at the University of Chicago who has studied representational similarity in language models .
“如果你只测试一个数据集，你不一定知道[结果]如何概括，”芝加哥大学研究语言模型中表征相似性的研究员 Christopher Wolfram 说。
“Who knows what would happen if you did some weirder dataset?”
“谁知道如果你做了一些更奇怪的数据集会发生什么？”
Isola acknowledged that the issue is far from settled.
伊索拉承认这个问题远未解决。
It’s not a question that any one paper can resolve: In principle, you can measure models’ representations of any picture or any sentence.
这不是任何一篇论文就能解决的问题：原则上，你可以测量模型对任何图片或任何句子的表示。
To him, cases where models do exhibit convergence are more compelling than cases where they may not.
对他来说，模型确实表现出收敛性的案例比模型可能不表现出收敛性的案例更引人注目。
Alexei Efros argues that the differences between AI models reveal more than the similarities.
阿列克谢·埃夫罗斯 (Alexei Efros) 认为，人工智能模型之间的差异比相似之处更多。
“The endeavor of science is to find the universals,” Isola said.
“科学的努力就是寻找共性，”伊索拉说。
“We could study the ways in which models are different or disagree, but that somehow has less explanatory power than identifying the commonalities.”
“我们可以研究模型不同或不一致的方式，但这在某种程度上比确定共性的解释力要弱。”
Other researchers argue that it’s more productive to focus on where models’ representations differ.
其他研究人员认为，关注模型表征的不同之处会更有成效。
Among them is Alexei Efros , a researcher at the University of California, Berkeley, who has been an adviser to three of the four members of the MIT team.
其中包括加州大学伯克利分校研究员阿列克谢·埃夫罗斯（Alexei Efros），他曾担任麻省理工学院团队四名成员中三人的顾问。
“They’re all good friends and they’re all very, very smart people,” Efros said.
“他们都是好朋友，而且都是非常非常聪明的人，”埃弗罗斯说。
“I think they’re wrong, but that’s what science is about.”
“我认为他们错了，但这就是科学的意义。”
Efros noted that in the Wikipedia dataset that Huh used, the images and text contained very similar information by design.
Efros 指出，在 Huh 使用的维基百科数据集中，图像和文本在设计上包含非常相似的信息。
But most data we encounter in the world has features that resist translation.
但我们在世界上遇到的大多数数据都具有难以翻译的特征。
“There is a reason why you go to an art museum instead of just reading the catalog,” he said.
“你去艺术博物馆而不是仅仅阅读目录是有原因的，”他说。
Any intrinsic sameness across models doesn’t have to be perfect to be useful.
模型之间任何内在的相同性不一定是完美的才有用。
Last summer, researchers devised a method to translate internal representations of sentences from one language model to another.
去年夏天，研究人员设计了一种将句子的内部表示从一种语言模型翻译成另一种语言模型的方法。
And if language and vision model representations are to some extent interchangeable, that could lead to new ways to train models that learn from both data types.
如果语言和视觉模型表示在某种程度上可以互换，这可能会带来训练从这两种数据类型学习的模型的新方法。
Isola and others explored one such training scheme in a recent paper.
伊索拉和其他人在最近的一篇论文中探讨了这样的培训计划。
Despite these promising developments, other researchers think it’s unlikely that any single theory will fully capture the behavior of modern AI models.
尽管取得了这些有希望的进展，但其他研究人员认为，任何单一理论都不太可能完全捕捉现代人工智能模型的行为。
“You can’t reduce a trillion-parameter system to simple explanations,” said Jeff Clune , an AI researcher at the University of British Columbia.
“你无法将万亿参数的系统简化为简单的解释，”不列颠哥伦比亚大学的人工智能研究员 Jeff Clune 说。
“The answers are going to be complicated.”
“答案将会很复杂。”
Get highlights of the most important news delivered to your email inbox
获取发送到您的电子邮件收件箱的最重要新闻的摘要
Quanta Magazine moderates comments to facilitate an informed, substantive, civil conversation.
《Quanta》杂志对评论进行审核，以促进知情、实质性、文明的对话。
Abusive, profane, self-promotional, misleading, incoherent or off-topic comments will be rejected.
辱骂、亵渎、自我推销、误导、语无伦次或偏离主题的评论将被拒绝。
Moderators are staffed during regular business hours (New York time) and can only accept comments written in English.
版主在正常工作时间（纽约时间）有工作人员，并且只能接受英文评论。