AI as the engine, humans as the steering wheel

Special thanks to Devansh Mehta, Davide Crapis and Julian Zawistowski for feedback and review, Tina Zhen, Shaw Walters and others for discussion.

特别感谢 Devansh Mehta、Davide Crapis 和 Julian Zawistowski 的反馈和审阅，感谢 Tinazhen、Shaw Walters 和其他人的讨论。

If you ask people what they like about democratic structures, whether governments, workplaces, or blockchain-based DAOs, you will often hear the same arguments: they avoid concentration of power, they give their users strong guarantees because there isn't a single person who can completely change the system's direction on a whim, and they can make higher-quality decisions by gathering the perspectives and wisdom of many people.

如果你问人们喜欢民主结构的什么，无论是政府、工作场所还​​是基于区块链的 DAO，你经常会听到相同的论点：它们避免权力集中，它们为用户提供强有力的保证，因为没有一个人可以随心所欲地完全改变系统的方向，并且它们可以通过收集许多人的观点和智慧来做出更高质量的决策。

If you ask people what they dislike about democratic structures, they will often give the same complaints: average voters are not sophisticated, because each voter only has a small chance of affecting the outcome, few voters put high-quality thought into their decisions, and you often get either low participation (making the system easy to attack) or de-facto centralization because everyone just defaults to trusting and copying the views of some influencer.

如果你问人们他们不喜欢民主结构的什么，他们通常会给出同样的抱怨：普通选民并不成熟，因为每个选民影响结果的机会很小，很少有选民将高质量的想法纳入他们的决策中，而且你经常会得到低参与度（使系统容易受到攻击）或事实上的中心化，因为每个人都默认信任和复制某些影响者的观点。

The goal of this post will be to explore a paradigm that could perhaps use AI to get us the benefits of democratic structures without the downsides.

这篇文章的目标是探索一种范例，或许可以利用人工智能让我们从民主结构中获益，而不会带来负面影响。

" AI as the engine, humans as the steering wheel ".

“人工智能为引擎，人类为方向盘”。

Humans provide only a small amount of information into the system, perhaps only a few hundred bits, but each of those bits is a well-considered and very high-quality bit.

人类只向系统提供少量信息，可能只有几百位，但每一个位都是经过深思熟虑且质量非常高的。

AI treats this data as an "objective function", and tirelessly makes a very large number of decisions doing a best-effort at fitting these objectives.

人工智能将这些数据视为“目标函数”，并不知疲倦地做出大量决策，尽最大努力实现这些目标。

In particular, this post will explore an interesting question: can we do this without enshrining a single AI at the center, instead relying on a competitive open market that any AI (or human-AI hybrid) is free to participate in?

特别是，这篇文章将探讨一个有趣的问题：我们是否可以在不以单一人工智能为中心的情况下实现这一目标，而是依赖任何人工智能（或人类与人工智能混合体）都可以自由参与的竞争性开放市场？

The easiest way to insert human preferences into an AI-based mechanism is to make a single AI model, and have humans feed their preferences into it somehow.

将人类偏好插入基于人工智能的机制的最简单方法是创建一个单一的人工智能模型，并让人类以某种方式将他们的偏好输入其中。

There are easy ways to do this: you can just put a text file containing a list of people's instructions into the system prompt.

有一些简单的方法可以做到这一点：您可以将包含人员指令列表的文本文件放入系统提示符中。

Then you use one of many "agentic AI frameworks" to give the AI the ability to access the internet, hand it the keys to your organization's assets and social media profiles, and you're done.

然后，您使用众多“代理人工智能框架”之一来赋予人工智能访问互联网的能力，将您组织的资产和社交媒体资料的密钥交给它，然后就完成了。

After a few iterations, this may end up good enough for many use cases, and I fully expect that in the near future we are going to see many structures involving AIs reading instructions given by a group (or even real-time reading a group chat) and taking actions as a result.

经过几次迭代后，这对于许多用例来说可能已经足够好了，我完全预计在不久的将来我们将看到许多涉及人工智能读取群组给出的指令（甚至实时读取群组聊天）并采取行动的结构。

Where this structure is not ideal is as a governing mechanism for long-lasting institutions.

这种结构不理想的地方在于作为持久机构的治理机制。

One valuable property for long-lasting institutions to have is credible neutrality .

持久的机构所拥有的一项宝贵财产是可信的中立性。

In my post introducing this concept, I listed four properties that are valuable for credible neutrality:

在我介绍这个概念的文章中，我列出了对于可信中立有价值的四个属性：

An LLM (or AI agent) satisfies 0/4.

LLM（或 AI 代理）满足 0/4。

The model inevitably has a huge amount of specific people and outcome preferences encoded through its training process.

该模型不可避免地会在训练过程中编码大量特定的人和结果偏好。

Sometimes this leads to the AI having preferences in surprising directions, eg.

有时这会导致人工智能在令人惊讶的方向上有偏好，例如。

see this recent research suggesting that major LLMs value lives in Pakistan far more highly than lives in the USA (!!).

请参阅最近的研究表明，主要的法学硕士对巴基斯坦生活的重视程度远远高于美国的生活（！！）。

It can be open-weights , but that's far from open-source ; we really don't know what devils are hiding in the depths of a model.

它可以是开放权重的，但这离开源还很远；我们真的不知道模型深处隐藏着什么魔鬼。

It's the opposite of simple: the Kolmogorov complexity of an LLM is in the tens of billions of bits, about the same as that of all US law (federal + state + local) put together .

它与简单相反：法学硕士的柯尔莫哥洛夫复杂度为数百亿位，大约相当于所有美国法律（联邦+州+地方）的总和。

And because of how rapidly AI is evolving, you'll have to change it every three months.

由于人工智能的发展速度非常快，你必须每三个月改变一次。

For this reason, an alternative approach that I favor exploring for many use cases is to make a simple mechanism be the rules of the game, and let AIs be the players .

出于这个原因，我赞成在许多用例中探索的另一种方法是让一个简单的机制成为游戏规则，让人工智能成为玩家。

This is the same insight that makes markets so effective: the rules are a relatively dumb system of property rights, with edge cases decided by a court system that slowly accumulates and adjusts precedents, and all of the intelligence comes from entrepreneurs operating "at the edge".

这正是使市场如此有效的洞察力：规则是一个相对愚蠢的产权制度，边缘案件由慢慢积累和调整先例的法院系统决定，所有的情报都来自于“边缘”运作的企业家。

The individual "game players" can be LLMs, swarms of LLMs interacting with each other and calling into various internet services, various AI + human combinations, and many other constructions; as a mechanism designer, you do not need to know.

个体“游戏玩家”可以是法学硕士、法学硕士群体，彼此互动并调用各种互联网服务、各种人工智能+人类组合以及许多其他结构；作为机制设计者，你不需要知道。

The ideal goal is to have a mechanism that functions as an automaton - if the goal of the mechanism is choosing what to fund, then it should feel as much as possible like Bitcoin or Ethereum block rewards.

理想的目标是拥有一个像自动机一样运行的机制——如果该机制的目标是选择资助什么，那么它应该尽可能像比特币或以太坊区块奖励。

The goal of the steering mechanism is to provide a faithful representation of the participants' underlying goals.

指导机制的目标是忠实地体现参与者的基本目标。

It only needs to provide a small amount of information, but it should be high-quality information.

它只需要提供少量的信息，但应该是高质量的信息。

You can think of the mechanism as exploiting an asymmetry between coming up with an answer and verifying the answer.

您可以将该机制视为利用提出答案和验证答案之间的不对称性。

This is similar to how a sudoku is difficult to solve, but it's easy to verify that a solution is correct .

这类似于数独很难解决，但很容易验证解决方案是否正确。

You (i) create an open market of players to act as "solvers", and then (ii) maintain a human-run mechanism that performs the much simpler task of verifying solutions that have been presented.

您（i）创建一个开放的玩家市场来充当“解决者”，然后（ii）维护一个人工运行的机制，该机制执行验证已提出的解决方案的简单得多的任务。

Futarchy was originally introduced by Robin Hanson as " vote values, but bet beliefs ".

Futarchy 最初由 Robin Hanson 介绍为“投票价值，但赌注信仰”。

A voting mechanism chooses a set of goals (which can be anything, with the caveat that they need to be measurable) which get combined into a metric M.

投票机制选择一组目标（可以是任何目标，但需要注意的是它们需要可测量），这些目标组合成一个度量 M。

When you need to make a decision (for simplicity, let's say it's YES/NO), you set up conditional markets : you ask people to bet on (i) whether YES or NO will be chosen, (ii) value of M if YES is chosen, otherwise zero, (iii) value of M if NO is chosen, otherwise zero.

当您需要做出决定时（为了简单起见，假设是“是”/“否”），您可以设置条件市场：您要求人们下注（i）是否会选择“是”或“否”，（ii）如果选择“是”，则M的值，否则为零，（iii）如果选择“否”，则M的值，否则为零。

Given these three variables, you can figure out if the market thinks YES or NO is more bullish for the value of M.

考虑到这三个变量，您可以弄清楚市场是否认为“是”或“否”对 M 的值更看好。

"Price of the company share" (or, for a cryptocurrency, a token) is the most commonly cited metric, because it's so easy to understand and measure, but the mechanism can support many kinds of metrics: monthly active users, median self-reported happiness of some group of constituents, some quantifiable measure of decentralization, etc.

“公司股票的价格”（或者，对于加密货币来说，是代币）是最常被引用的指标，因为它很容易理解和衡量，但该机制可以支持多种指标：每月活跃用户、某些群体的自我报告幸福感中位数、一些可量化的去中心化衡量标准等。

Futarchy was originally invented in the pre-AI era.

Futarchy 最初是在前人工智能时代发明的。

However, futarchy fits very naturally in the "sophisticated solver, easy verifier" paradigm described in the previous section, and traders in a futarchy can be AI (or human+AI combinations) too.

然而，futarchy 非常自然地适合上一节中描述的“复杂的求解器，简单的验证器”范例，并且 futarchy 中的交易者也可以是人工智能（或人类+人工智能的组合）。

The role of the "solvers" (prediction market traders) is to determine how each proposed plan will affect the value of a metric in the future.

“解决者”（预测市场交易者）的作用是确定每个提议的计划将如何影响未来指标的价值。

This is hard.

这很难。

The solvers make money if they are right, and lose money if they are wrong.

如果解决者是对的，他们就会赚钱；如果他们错了，他们就会赔钱。

The verifiers (the people voting on the metric, adjusting the metric if they notice that it is being "gamed" or is otherwise becoming outdated, and determining the actual value of the metric at some future time) need only answer the simpler question "what is the value of the metric now?"

验证者（对指标进行投票的人，如果发现指标被“欺骗”或过时了，则调整指标，并在未来某个时间确定指标的实际值）只需要回答更简单的问题“现在指标的价值是多少？”

Distilled human judgement is a class of mechanisms that works as follows.

人类的精炼判断是一类机制，其工作原理如下。

There is a very large number (think: 1 million) of questions that need to be answered .

有大量（例如：100 万个）问题需要回答。

Natural examples include:

自然的例子包括：

You have a jury that can answer such questions, though at the cost of spending a lot of effort on each answer.

你有一个陪审团可以回答这些问题，尽管代价是在每个答案上花费大量精力。

You ask the jury to only a small number of the questions (eg.

您只向陪审团提出一小部分问题（例如：

if the total list has 1 million items, the jury perhaps only provides answers on 100 of them).

如果总清单有 100 万个项目，评审团可能只提供其中 100 个的答案）。

You can even ask the jury indirect questions: instead of asking "what percent of total credit does Alice deserve?", you can ask "does Alice or Bob deserve more credit, and how many times more?".

您甚至可以向陪审团提出间接问题：您可以问“Alice 或 Bob 是否应该得到更多的功劳，以及多少倍？”，而不是问“Alice 应得的总功劳百分比是多少？”。

When designing the jury mechanism, you can reuse time-tested mechanisms from the real world like grants committees, courts (determining value of a judgement), appraisals, etc, though of course the jury participants are themselves welcome to use new-fangled AI research tools to help them come to an answer.

在设计陪审团机制时，你可以重用现实世界中经过时间考验的机制，如拨款委员会、法院（确定判决的价值）、评估等，当然，陪审团参与者本身也欢迎使用新奇的人工智能研究工具来帮助他们找到答案。

You then allow anyone to submit a list of numerical responses to the entire set of questions (eg.

然后，您允许任何人提交对整组问题的数字回答列表（例如，

providing an estimate for how much credit each participant in the entire list deserves).

提供对整个列表中每个参与者应得的信用的估计）。

Participants are encouraged to use AI to do this, though they can use any technique: AI, human-AI hybrid, AI with access to internet search and the ability to autonomously hire other human or AI workers, cybernetically enhanced monkeys, etc.

鼓励参与者使用人工智能来做到这一点，尽管他们可以使用任何技术：人工智能、人类-人工智能混合体、能够访问互联网搜索并能够自主雇用其他人类或人工智能工人的人工智能、控制论增强的猴子等。

Once the full-list providers and the jurors have both submitted their answers, the full lists are checked against the jury answers, and some combination of the full lists that are most compatible with the jury answers is taken as the final answer .

一旦完整列表提供者和陪审员都提交了他们的答案，完整列表将与陪审团答案进行检查，并且与陪审团答案最相符的完整列表的某种组合将被视为最终答案。

The distilled human judgement mechanism is different from futarchy, but has some important similarities:

精炼的人类判断机制与 futarchy 不同，但有一些重要的相似之处：

Toy example of distilled human judgement for credit assignment, see python code here .

用于信用分配的人类精炼判断的玩具示例，请参阅此处的 python 代码。

The script asks you to be the jury, and contains some AI-generated (and human-generated) full lists pre-included in the code.

该脚本要求您担任陪审团，并包含一些预先包含在代码中的人工智能生成（和人类生成）的完整列表。

The mechanism identifies the linear combination of full lists that best-fits the jury answers.

该机制识别出最适合陪审团答案的完整列表的线性组合。

In this case, the winning combination is 0.199 * Claude's answer + 0.801 * Deepseek's answer; this combination matches the jury answers better than any single model does.

在这种情况下，获胜组合是 0.199 * Claude 的答案 + 0.801 * Deepseek 的答案；这种组合比任何单一模型都更符合陪审团的答案。

These coefficients would also be the rewards given to the submitters.

这些系数也将是给予提交者的奖励。

The "humans as a steering wheel" aspect in this "defeating Sauron" example is reflected in two places.

这个“击败索伦”例子中的“人类作为方向盘”方面体现在两个地方。

First, there is high-quality human judgement being applied on each individual question, though this is still leveraging the jury as "technocratic" evaluators of performance.

首先，对每个单独的问题都应用了高质量的人类判断，尽管这仍然利用陪审团作为绩效的“技术官僚”评估者。

Second, there is an implied voting mechanism that determines if "defeating Sauron" is even the right goal (as opposed to, say, trying to ally with him, or offering him all the territory east of some critical river as a concession for peace).

其次，有一个隐含的投票机制来决定“击败索伦”是否是正确的目标（而不是试图与他结盟，或者向他提供一些关键河流以东的所有领土作为和平的让步）。

There are other distilled human judgement use cases where the jury task is more directly values-laden: for example, imagine a decentralized social media platform (or sub-community) where the jury's job is to label randomly selected forum posts as following or not following the community's rules.

还有其他精炼的人类判断用例，其中陪审团的任务更直接地承载价值观：例如，想象一个去中心化的社交媒体平台（或子社区），其中陪审团的工作是将随机选择的论坛帖子标记为遵循或不遵循社区的规则。

There are a few open variables within the distilled human judgement paradigm:

在精炼的人类判断范式中存在一些开放变量：

In general, the goal is to take human judgement mechanisms that are known to be effective and bias-minimizing and have stood the test of time (eg.

一般来说，目标是采用已知有效、偏见最小化并经受住时间考验的人类判断机制（例如，

think of how the adversarial structure of a court system includes both the two parties to a dispute, who have high information but are biased, and a judge, who has low information but is probably unbiased), and use an open market of AIs as a reasonably high-fidelity and very low-cost predictor of these mechanisms (this is similar to how "distillation" of LLMs works).

想想法院系统的对抗性结构如何包括争议双方（信息量高但有偏见）和法官（信息量低但可能没有偏见），并使用人工智能的开放市场作为这些机制的合理高保真度和非常低成本的预测器（这类似于法学硕士“蒸馏”的工作原理）。

Deep funding is the application of distilled human judgement to the problem of filling in the weights of edges on a graph representing "what percent of the credit for X belongs to Y?"

深度融资是将人类判断的精炼应用到填充图上边的权重的问题上，该图表示“X 的功劳中有多少百分比属于 Y？”

Output of two-level deep funding example: the ideological origins of Ethereum.

两级深度资金输出示例：以太坊的思想起源。

See python code here .

请参阅此处的 python 代码。

Here, the goal is to distribute the credit for philosophical contributions that led to Ethereum.

在这里，我们的目标是分配以太坊的哲学贡献的功劳。

Let's look at an example:

让我们看一个例子：

This approach is designed to work in domains where work is built on top of previous work and the structure of this is highly legible.

这种方法旨在在工作建立在以前的工作之上并且其结构非常清晰的领域中工作。

Academia (think: citation graphs) and open source software (think: library dependencies and forking) are two natural examples.

学术界（想想：引用图）和开源软件（想想：库依赖关系和分叉）是两个自然的例子。

The goal of a well-functioning deep funding system would be to create and maintain a global graph, where any funder that is interested in supporting one particular project would be able to send funds to an address representing that node, and funds would automatically propagate to its dependencies (and recursively to their dependencies etc) based on the weights on the edges of the graph.

运行良好的深度融资系统的目标是创建和维护一个全局图，其中任何有兴趣支持某个特定项目的资助者都能够将资金发送到代表该节点的地址，并且资金将根据图边缘的权重自动传播到其依赖项（并递归到其依赖项等）。

You could imagine a decentralized protocol using a built-in deep funding gadget to issue its token : some in-protocol decentralized governance would choose a jury, and the jury would run the deep funding mechanism, as the protocol automatically issues tokens and deposits them into the node corresponding to itself.

你可以想象一个去中心化协议使用内置的深度融资小工具来发行其代币：一些协议内的去中心化治理会选择一个陪审团，陪审团将运行深度融资机制，因为协议会自动发行代币并将其存入与其自身对应的节点。

By doing so, the protocol rewards all of its direct and indirect contributors in a programmatic way reminiscent of how Bitcoin or Ethereum block rewards rewarded one specific type of contributor (miners).

通过这样做，该协议以编程方式奖励所有直接和间接贡献者，让人想起比特币或以太坊区块奖励如何奖励一种特定类型的贡献者（矿工）。

By influencing the weights of the edges, the jury gets a way to continuously define what types of contributions it values.

通过影响边的权重，陪审团可以不断定义其重视的贡献类型。

This mechanism could function as a decentralized and long-term-sustainable alternative to mining, sales or one-time airdrops.

这一机制可以作为挖矿、销售或一次性空投的去中心化且长期可持续的替代方案。

Often, making good judgements on questions like those in the examples above requires having access to private information: an organization's internal chat logs, information confidentially submitted by community members, etc. One benefit of "just using a single AI", especially for smaller-scale contexts, is that it's much more acceptable to give one AI access to the information than to make it public for everyone.

通常，要对上述示例中的问题做出良好的判断，需要访问私人信息：组织的内部聊天日志、社区成员秘密提交的信息等。“仅使用单个人工智能”的好处之一，特别是对于较小规模的环境，是让一个人工智能访问信息比向所有人公开信息更容易被接受。

To make distilled human judgement or deep funding work in these contexts, we could try to use cryptographic techniques to securely give AIs access to private information.

为了在这些背景下进行精炼的人类判断或深度资助工作，我们可以尝试使用加密技术来安全地让人工智能访问私人信息。

The idea is to use multi-party computation (MPC), fully homomorphic encryption (FHE), trusted execution environments (TEEs) or similar mechanisms to make the private information available, but only to mechanisms whose only output is a "full list submission" that gets directly put into the mechanism.

这个想法是使用多方计算（MPC）、完全同态加密（FHE）、可信执行环境（TEE）或类似的机制来使私有信息可用，但仅限于其唯一输出是直接放入该机制的“完整列表提交”的机制。

If you do this, then you would have to restrict the set of mechanisms to just being AI models (as opposed to humans or AI + human combinations, as you can't let humans see the data), and in particular models running in some specific substrate (eg.

如果你这样做，那么你将不得不将机制集限制为人工智能模型（而不是人类或人工智能+人类组合，因为你不能让人类看到数据），特别是在某些特定基质中运行的模型（例如。

MPC, FHE, trusted hardware).

MPC、FHE、可信硬件）。

A major research direction is figuring out near-term practical versions of this that are efficient enough to make sense.

一个主要的研究方向是找出近期实用的版本，这些版本足够有效且有意义。

Designs like this have a number of promising benefits.

像这样的设计有许多有前途的好处。

By far the most important one is that they allow for the construction of DAOs where human voters are in control of setting the direction, but they are not overwhelmed with an excessively large number of decisions to make .

到目前为止，最重要的一点是，它们允许构建 DAO，让人类选民可以控制设定方向，但他们不会因需要做出过多的决策而不知所措。

They hit the happy medium where each person doesn't have to make N decisions, but they have more power than just making one decision (how delegation typically works), and in a way that is more capable of eliciting rich preferences that are difficult to express directly.

他们找到了一个折衷方案，即每个人不必做出 N 个决定，但他们比只做出一个决定（授权通常如何运作）拥有更多的权力，并且以一种更能够引发难以直接表达的丰富偏好的方式。

Additionally, mechanisms like this seem to have an incentive smoothing property.

此外，像这样的机制似乎具有激励平滑特性。

What I mean here by "incentive smoothing" is a combination of two factors:

我在这里所说的“激励平滑”是两个因素的组合：

The terms confusion and diffusion here are taken from cryptography , where they are key properties of what makes ciphers and hash functions secure.

这里的术语“混淆”和“扩散”取自密码学，它们是密码和哈希函数安全的关键属性。

A good example of incentive smoothing in the real world today is the rule of law: the top level of the government does not regularly take actions of the form "give Alice's company $200M", "fine Bob's company $100M", etc, rather it passes rules that are intended to apply evenly to large sets of actors, which then get interpreted by a separate class of actors.

当今现实世界中激励平滑的一个很好的例子是法治：政府高层不会定期采取“给爱丽丝的公司 2 亿美元”、“罚款鲍勃的公司 1 亿美元”等形式的行动，而是通过旨在均匀适用于大量参与者的规则，然后由单独的参与者类别进行解释。

When this works, the benefit is that it greatly reduces the benefits of bribery and other forms of corruption.

当这一点发挥作用时，好处是大大减少了贿赂和其他形式腐败的好处。

And when it's violated (as it often is in practice), those issues quickly become greatly magnified.

当它被违反时（正如实践中经常发生的那样），这些问题很快就会被大大放大。

AI is clearly going to be a very large part of the future, and this will inevitably include being a large part of the future of governance.

人工智能显然将成为未来的重要组成部分，这也将不可避免地成为未来治理的重要组成部分。

However, if you are involving AI in governance, this has obvious risks: AI has biases, it could be intentionally corrupted during the training process, and AI technology is evolving so quickly that "putting an AI in charge" may well realistically mean "putting whoever is responsible for upgrading the AI in charge" .

然而，如果让人工智能参与治理，这就存在明显的风险：人工智能存在偏见，在训练过程中可能会被故意破坏，而且人工智能技术发展如此之快，“让人工智能负责”很可能实际上意味着“让负责升级人工智能的人负责”。

Distilled human judgement offers an alternative path forward, which lets us harness the power of AI in an open free-market way while keeping a human-run democracy in control.

经过提炼的人类判断提供了一条替代的前进道路，它让我们能够以开放的自由市场方式利用人工智能的力量，同时保持人类运行的民主的控制。

Anyone interested in more deeply exploring and participating in these mechanisms today is highly encouraged to check out the currently active deep funding round at https://cryptopond.xyz/modelfactory/detail/2564617 .

强烈建议任何有兴趣更深入探索和参与这些机制的人在 https://cryptopond.xyz/modelfactory/detail/2564617 上查看当前活跃的深度融资轮次。