Shaping global governance of AI agents

Artificial intelligence (AI) agents—technology that plans and executes tasks autonomously —have accelerated sharply over the past two years in both capability and projected market size.

人工智能（AI）代理——自主计划和执行任务的技术——在过去两年中在能力和预计市场规模方面都急剧加速。

In 2025, agents improved their accuracy in computer tasks from 12% to 66.3% , lagging the average human baseline accuracy by just 6 points.

2025 年，智能体在计算机任务中的准确率从 12% 提高到 66.3%，仅落后人类平均基线准确率 6 个百分点。

In terms of market size, Precedence Research estimates that the market will grow from $8 billion in 2025 to $199 billion by 2034.

就市场规模而言，Precedence Research 预计该市场将从 2025 年的 80 亿美元增长到 2034 年的 1990 亿美元。

Such rapid acceleration implies that existing governance models for AI more generally might not be sufficient.

如此快速的加速意味着现有的人工智能治理模型可能还不够。

The shift from static model oversight to dynamic system governance raises new challenges around control, speed, traceability, and human oversight.

从静态模型监督到动态系统治理的转变在控制、速度、可追溯性和人工监督方面提出了新的挑战。

That distinction matters.

这种区别很重要。

The central governance question is no longer only what the model said.

中央治理问题不再只是模型所说的。

Increasingly, it is what the system did, why it was allowed to do it, and who is accountable.

越来越多的问题是系统做了什么、为什么允许这样做以及谁负责。

How, then, can governance mechanisms be built to seize the opportunities of agentic AI and mitigate its new and complex risks?

那么，如何建立治理机制来抓住代理人工智能的机遇并减轻其新的复杂风险呢？

These issues were at the center of the May 2026 World Economic Forum panel at the AI+Expo in Washington, D.C. A key theme was that risk-based approaches, life-cycle governance, accountability, safety, and human oversight remain relevant but must become far more operational as systems act faster and with less direct human intervention.

这些问题是 2026 年 5 月在华盛顿特区举行的人工智能+博览会上举行的世界经济论坛小组讨论的中心议题。一个关键主题是基于风险的方法、生命周期治理、问责制、安全和人类监督仍然具有相关性，但必须变得更具可操作性，因为系统的行动速度更快且直接的人为干预更少。

AI agents are proving to support productivity, public services, education, health care, translation, accessibility, growth of small- and medium-sized enterprises, and development goals across both the Global North and the Global South.

事实证明，人工智能代理可以支持生产力、公共服务、教育、医疗保健、翻译、可及性、中小企业的发展以及全球北方和全球南方的发展目标。

Already, they are compressing research timelines for pharmaceutical companies, optimizing industrial goods supply chains, and reducing costs of customer service.

他们已经在压缩制药公司的研究时间、优化工业品供应链并降低客户服务成本。

In manufacturing , agents have moved from simply flagging a defect to analyzing the systems, finding the probable cause, and removing the defective unit.

在制造业中，代理已经从简单地标记缺陷转向分析系统、查找可能的原因并删除有缺陷的单元。

These agents do not require large models or the newest chips, which makes them more accessible to lower-income countries than full digital transformations are.

这些代理不需要大型模型或最新的芯片，这使得它们比全面数字化转型更容易被低收入国家所接受。

In the public sector, a World Economic Forum study of 70 core government functions found public services among the highest-readiness targets for agents but cautioned that information-driven tasks should come before judgment-based ones such as eligibility assessment, where agents would affect sensitive decisions about individuals.

在公共部门，世界经济论坛对 70 个核心政府职能的研究发现，公共服务是代理人的最高准备目标之一，但警告说，信息驱动的任务应优先于基于判断的任务，例如资格评估，因为代理人会影响有关个人的敏感决策。

Early returns are promising even from simpler deployments: The U.S. Centers for Disease Control and Prevention estimates a 527% return on its staffwide AI chatbot.

即使是更简单的部署，早期回报也很有希望：美国疾病控制与预防中心估计，其全员人工智能聊天机器人的回报率为 527%。

With good governance of AI agents, agencies can leverage their strengths to improve productivity, complete complex workflow automation, and respond to crises with rapid needs assessments.

通过对人工智能代理的良好治理，机构可以利用其优势来提高生产力，完成复杂的工作流程自动化，并通过快速需求评估来应对危机。

Yet documented evidence available today on deployed value is largely pre-agentic, and the governance question is arriving ahead of the deployments that will test it.

然而，目前关于部署价值的记录证据很大程度上是前代理的，治理问题是在测试它的部署之前出现的。

So far, adoption remains dominated by knowledge workers in higher-GDP countries , and enterprise uptake trails consumer use as compliance, budgets, and internal rules slow deployment.

到目前为止，采用仍然由 GDP 较高国家的知识工作者主导，而企业采用落后于消费者使用，因为合规性、预算和内部规则部署缓慢。

A recent Microsoft analysis of 100,000 Copilot chats found that almost half of the requests were for the agent to support cognitive work, while the remaining half comprised working with people, finding information, and producing work.

微软最近对 100,000 个 Copilot 聊天进行的分析发现，几乎一半的请求是要求座席支持认知工作，而剩下的一半则包括与人合作、查找信息和生产工作。

These usage patterns are an early indication of where value is concentrating.

这些使用模式是价值集中的早期迹象。

Agents combine autonomy and the ability to operate independently with the authority, permissions, and access rights that allow them to act within defined boundaries.

代理将自主性和独立操作的能力与允许他们在定义的边界内行动的权限、许可和访问权结合起来。

Autonomy, adaptability, and complexity are the cornerstones of agentic systems and also the characteristics that make effective governance difficult.

自主性、适应性和复杂性是代理系统的基石，也是导致有效治理困难的特征。

AI agents use complex machine-learning algorithms to analyze probabilities and determine actions in a way that is not necessarily possible for humans to understand and therefore audit.

人工智能代理使用复杂的机器学习算法来分析概率并以人类不一定能够理解并因此审计的方式确定行动。

Existing governance structures are based on discrete steps and stages, moving from model development to procurement, deployment, and output review.

现有的治理结构基于离散的步骤和阶段，从模型开发到采购、部署和输出审查。

Yet AI agents make those boundaries less meaningful due to the combination of models with memory, orchestration, tools, and access to external systems which leads to a completely different risk profile.

然而，由于模型与内存、编排、工具以及对外部系统的访问相结合，人工智能代理使这些边界变得不那么有意义，从而导致完全不同的风险状况。

For example, traditional AI chatbots can be reviewed by humans before action is taken, but AI agents are structured specifically to invoke another system or action before a human intervenes.

例如，传统的人工智能聊天机器人可以在采取行动之前由人类进行审查，但人工智能代理经过专门设计，可以在人类干预之前调用另一个系统或行动。

Once agents are connected to tools and granted authority, they become consequential in a way that conventional software controls are not equipped to manage, leading to risks like goal misalignment, behavioral drift, tool misuse, and coordination failure.

一旦代理连接到工具并被授予权限，它们就会以传统软件控制无法管理的方式产生后果，从而导致目标不一致、行为漂移、工具滥用和协调失败等风险。

They thus start to resemble delegated decisionmakers , requiring much stronger oversight to deal with the expanded capabilities.

因此，他们开始类似于授权决策者，需要更强有力的监督来应对扩展的能力。

A governance gap often opens when organizations know what an agent do, but do not necessarily know what they should allow it to do.

当组织知道代理做什么，但不一定知道他们应该允许代理做什么时，常常会出现治理差距。

Even when organizations create the strongest systems, they remain far from uniformly reliable, as even a properly authorized agent can fail , whether its misunderstanding a task, failing to recover after an unexpected event, or increased vulnerabilities after integrating a tool.

即使组织创建了最强大的系统，它们仍然远非一致可靠，因为即使是经过适当授权的代理也可能会失败，无论是误解任务、在意外事件后无法恢复，还是集成工具后漏洞增加。

A more complete governance model should assess at least five interacting factors, including capability, authority, architecture, context, and consequence.

更完整的治理模型应该评估至少五个相互作用的因素，包括能力、权限、架构、上下文和结果。

Models can differ in risk levels based on different applications and stages.

根据不同的应用和阶段，模型的风险级别可能有所不同。

For example, a model could be low risk during actions like retrieving information, but it could turn higher risk when modifying records, communicating, or initiating transactions.

例如，模型在检索信息等操作期间可能风险较低，但在修改记录、通信或发起交易时可能会带来较高风险。

A helpful tool to deal with this reality is the Agent Capability and Authorization Profile developed by the World Economic Forum.

应对这一现实的一个有用工具是世界经济论坛开发的代理能力和授权配置文件。

This tool suggests that models, systems, and agent cards document what an agent can do, while a profile records what it has been authorized to do.

该工具建议模型、系统和代理卡记录代理可以执行的操作，而配置文件则记录其已被授权执行的操作。

This makes the scope, operating context, permissions, controls, evaluation evidence, monitoring, and ownership explicit for a specific deployment within a specific workflow.

这使得特定工作流程中特定部署的范围、操作上下文、权限、控制、评估证据、监控和所有权变得明确。

Each agent likely carries several profiles, so these records should complement, not substitute, technical evaluation of the entire underlying system.

每个代理可能都有多个配置文件，因此这些记录应该补充而不是替代整个底层系统的技术评估。

Governance structures for traditional AI are typically focused on controlling how humans interact with AI through measures such as content filtering, data protection, and output review.

传统人工智能的治理结构通常侧重于通过内容过滤、数据保护和输出审查等措施来控制人类与人工智能的交互方式。

Agents complicate this approach as governance now has to focus on how agents interact with systems and with one another.

代理使这一方法变得复杂，因为治理现在必须关注代理如何与系统以及彼此之间进行交互。

Stage-based models that are tied to design, procurement, or deployment will not be sufficient in the face of decentralized deployment and rising autonomy.

面对分散部署和不断增强的自主权，与设计、采购或部署相关的基于阶段的模型是不够的。

As every step in an agent’s chain of action accumulates new risk, governance must act continuously and at speed.

由于代理行动链中的每一步都会积累新的风险，治理必须持续、快速地采取行动。

Privacy and cybersecurity risks are critical, as attacks that manipulate the underlying data could lead to the agent making wrong and potentially dangerous decisions.

隐私和网络安全风险至关重要，因为操纵底层数据的攻击可能会导致代理做出错误且潜在危险的决策。

The data needed to power the agent, such as personal, enterprise, or system data, could also leak, exposing sensitive information.

为代理提供支持所需的数据（例如个人、企业或系统数据）也可能会泄漏，从而暴露敏感信息。

For example, prompt injection is the most common attack technique where one injection in a model can interfere with the entire system largely due to the architecture of agents where all inputs are treated as a unified token sequence, which makes it difficult to place boundaries or identify which part was interfered with.

例如，提示注入是最常见的攻击技术，其中模型中的一次注入可能会干扰整个系统，这很大程度上是由于代理的体系结构，其中所有输入都被视为统一的令牌序列，这使得很难放置边界或识别哪个部分受到干扰。

This leads to a lack of traceability and limited verification, which both point to the general challenge of the lack of human oversight that is possible.

这导致缺乏可追溯性和有限的验证，这两者都指向了可能缺乏人类监督的普遍挑战。

Within enterprises, new protocols like the model context protocol and agent-to-agent protocol are making agents more capable of interacting with one another across platforms without needing to integrate them.

在企业内部，模型上下文协议和代理到代理协议等新协议使代理更有能力跨平台相互交互，而无需集成它们。

While the increased coordination is promising in terms of productivity and capability, it requires more intentional governance as security attacks would become even more damaging with greater access to connected systems.

虽然加强协调在生产力和能力方面是有希望的，但它需要更加有意的治理，因为随着对互联系统的更多访问，安全攻击将变得更具破坏性。

Underlying all of this are accountability gaps between model developers, deployers, users, and affected communities.

所有这一切的背后是模型开发人员、部署人员、用户和受影响社区之间的责任差距。

Where communities are absent from the design of governance itself, harm becomes more likely whether it is intended or not.

如果社区没有参与治理本身的设计，那么无论有意无意，损害的可能性就更大。

As the Washington, D.C., panel highlighted, smaller, multilingual, open, or reusable models can support applications such as translation and accessibility with substantially less compute, which extends the opportunity well beyond the countries and companies able to deploy the largest systems.

正如华盛顿特区专家小组所强调的那样，更小的、多语言的、开放的或可重用的模型可以用更少的计算来支持翻译和辅助功能等应用程序，这将机会远远超出了能够部署最大系统的国家和公司。

Regulatory expertise, testing capacity, infrastructure, and bargaining power are unevenly distributed across countries and organizations, meaning global agent governance should not assume every deployer can negotiate risk allocation with major technology providers or access resources such as evaluation teams or extensive compute.

监管专业知识、测试能力、基础设施和议价能力在国家和组织之间分布不均，这意味着全球代理治理不应假设每个部署者都可以与主要技术提供商协商风险分配或访问评估团队或广泛计算等资源。

Unequal capacity globally thus poses significant risks, like that of traditional AI, where bias in the type and source of data they are built on can lead to agents that do not serve the interests or needs of certain communities.

因此，全球能力不平等带来了重大风险，就像传统人工智能一样，其所建立的数据类型和来源的偏见可能会导致代理无法满足某些社区的利益或需求。

In countries where there is limited regulatory capacity, risks related to AI agents specifically are amplified.

在监管能力有限的国家，与人工智能代理相关的风险尤其被放大。

In authoritarian contexts , agentic AI could be used with malicious intent to track opposition or push disinformation.

在独裁背景下，代理人工智能可能会被恶意用来追踪反对派或散布虚假信息。

Within and across countries, reskilling and labor force disruptions are amplified with agentic AI, especially for those in coding or customer service who are facing rapid automation by this technology.

在国家内部和国家之间，代理人工智能会加剧重新技能和劳动力中断，特别是对于那些面临该技术快速自动化的编码或客户服务人员来说。

Governance needs to be robust enough to protect users and practical enough to work in lower-capacity environments.

治理需要足够强大以保护用户，并且足够实用以在低容量环境中工作。

To ensure governance frameworks are up to the challenge to effectively regulate AI agents, they should keep existing principles that are working well but make them practical, measurable, and enforceable.

为了确保治理框架能够应对有效监管人工智能代理的挑战，他们应该保留行之有效的现有原则，但要使其实用、可衡量和可执行。

For example, the Hiroshima AI Process —the G7-led initiative for international guiding principles—and other risk-based, life-cycle governance approaches still apply, but the main gap is the implementation: turning principles into operational controls.

例如，广岛人工智能进程——七国集团主导的国际指导原则倡议——以及其他基于风险的生命周期治理方法仍然适用，但主要差距在于实施：将原则转化为操作控制。

Rather than relying on a “regulate and forget” mindset, adaptation and learning will be the foundation.

适应和学习将是基础，而不是依赖“监管和忘记”的心态。

Specifically, the public and private sector alike should:

具体而言，公共和私营部门都应：

Structured authorization models should name the agent’s owner and grant only necessary permissions .

结构化授权模型应指定代理的所有者并仅授予必要的权限。

Scope limits and the actions requiring human intervention need to be set in advance, so that high-risk actions are determined preemptively.

需要预先设定范围限制和需要人为干预的操作，以便预先确定高风险操作。

Two ways to do this are to implement guardrails and constraints.

实现此目的的两种方法是实施护栏和约束。

Guardrails stop actions, through input validation, content filtering, and scope enforcement.

护栏通过输入验证、内容过滤和范围强制来停止操作。

Constraints bound them through data boundaries, output limits, cost and time ceilings, and action restrictions.

通过数据边界、输出限制、成本和时间上限以及行动限制来约束他们。

Continuous, traceable evidence is equally important for mapping agents to their scope and actions.

连续、可追踪的证据对于将代理映射到其范围和行动同样重要。

Already, the EU AI Act , the National Institute of Standards and Technology (NIST)’s AI Risk Management Framework , and the OpenID Foundation’s October 2025 white paper on agentic identity all point toward requiring traceable evidence, which must continue to be enforced and updated as agents evolve.

欧盟人工智能法案、美国国家标准与技术研究所 (NIST) 的人工智能风险管理框架以及 OpenID 基金会 2025 年 10 月关于代理身份的白皮书都指出需要可追溯的证据，随着代理的发展，这些证据必须继续得到执行和更新。

End-to-end tracing, centralized gateways, and shared dashboards are important tools to improve visibility of agent errors, fallback triggers, and policy blocks for both deployers and developers.

端到端跟踪、集中式网关和共享仪表板是提高部署人员和开发人员对代理错误、回退触发器和策略块可见性的重要工具。

Runtime governance , which inspects agent actions across multiple inspection points, including continuous monitoring, logging, escalation, auditability, and redress, will be necessary to keep pace with the autonomous decisions happening in seconds.

运行时治理跨多个检查点检查代理操作，包括持续监控、日志记录、升级、可审计性和补救，这对于跟上几秒钟内发生的自主决策是必要的。

The EU AI Act places its heaviest obligations on the highest-risk categories, including health and employment, by requiring automatic logging and human oversight and backing those requirements with fines.

《欧盟人工智能法案》对健康和就业等最高风险类别规定了最重的义务，要求自动记录和人工监督，并通过罚款来支持这些要求。

However, these requirements are not yet in force.

然而，这些要求尚未生效。

The Digital Omnibus on AI, which entered into force in July, defers high-risk obligations to December 2027 for stand-alone systems and to August 2028 for AI embedded in regulated products.

人工智能数字综合计划于 7 月生效，将独立系统的高风险义务推迟到 2027 年 12 月，将受监管产品中嵌入的人工智能的高风险义务推迟到 2028 年 8 月。

Transparency and AI literacy duties were not postponed, but for roughly the next 16 months the controls governing agents that act in high-stakes domains will be organizational rather than statutory.

透明度和人工智能素养职责并未推迟，但在接下来的大约 16 个月内，对在高风险领域行事的管理机构的控制将是组织性的，而不是法定的。

Meanwhile the NIST AI RMF 1.0 , a co-created, voluntary U.S. framework, is based on four functions focused on cross-cutting policies and accountability, system-level risk context, quantitative and qualitative risk assessment, and risk treatment and incident response, noting that often the third function drives the most implementation challenges as a lack of data or infrastructure to do so causes failure that must be prioritized.

与此同时，NIST AI RMF 1.0（一个共同创建的美国自愿框架）基于四个职能，重点关注跨领域政策和问责制、系统级风险背景、定量和定性风险评估以及风险处理和事件响应，并指出，第三个职能通常会带来最大的实施挑战，因为缺乏数据或基础设施会导致必须优先考虑的失败。

Making the framework mandatory could accelerate its adoption implementation.

使该框架成为强制性的可以加速其采用实施。

Across the development and deployment cycles, better benchmarks and evaluations should be integrated, including trajectory tracking by observing decision chains and their interactions instead of only the output; factual grounding by using automated probes or rubric-based judges to see how and if actions are based on trusted sources; and adversarial red teaming by stress-testing models with realistic threats.

在整个开发和部署周期中，应该整合更好的基准和评估，包括通过观察决策链及其相互作用而不仅仅是输出来进行轨迹跟踪；通过使用自动调查或基于标准的法官来了解事实依据，以了解行动如何以及是否基于可信来源；以及通过具有现实威胁的压力测试模型进行对抗性红队。

Task success rate is a helpful benchmark which can be strengthened with more specific tracking under different scenarios and constraints.

任务成功率是一个有用的基准，可以通过在不同场景和约束下进行更具体的跟踪来加强它。

Customizable evaluation that is built into design will be key along with predeployment impact assessments .

设计中内置的可定制评估以及部署前影响评估将成为关键。

Sandboxes allow agent decisionmaking to be tested before deployment, and public-private partnerships, infrastructure investment, and capacity building can all reward up-front testing.

沙盒允许在部署之前测试代理决策，公私合作伙伴关系、基础设施投资和能力建设都可以奖励预先测试。

ISO/IEC 42001 is the first international management system standard for AI that is certifiable and supports readiness for the EU AI Act obligations without conferring presumption of conformity.

ISO/IEC 42001 是首个可认证的人工智能国际管理体系标准，支持为欧盟人工智能法案义务做好准备，但不授予合规性推定。

Singapore’s open-source AI Verify Toolkit offers an accessible way to test for fairness and robustness, and in January 2026, the country’s regulator released the Model AI Governance Framework for Agentic AI , a pioneering framework built specifically for agentic systems.

新加坡的开源 AI 验证工具包提供了一种测试公平性和鲁棒性的便捷方法，该国监管机构于 2026 年 1 月发布了 Agentic AI 的 AI 治理模型框架，这是一个专门为代理系统构建的开创性框架。

Where security risks can be managed, open standards such as Model Context Protocol support global interoperability.

在可以管理安全风险的地方，模型上下文协议等开放标准支持全球互操作性。

Establishing core safety and human-rights alignment targets can also help streamline cross-border operations so local systems meet the core standards, while leaving room for regional authorities to adjust their enforcement systems to local needs.

建立核心安全和人权协调目标还有助于简化跨境运营，使当地系统满足核心标准，同时为地区当局根据当地需求调整执法系统留出空间。

For example, Singapore’s AI Governance Framework acts as a hub for Southeast Asia and focuses on transparency and accountability while Africa’s AI Regional Hub Initiative collaborates with universities and focuses on ethics.

例如，新加坡的人工智能治理框架充当东南亚的中心，重点关注透明度和问责制，而非洲的人工智能区域中心计划则与大学合作，重点关注道德规范。

As agents continue to become more complex and widespread, the path forward for effective governance becomes more vital and necessary.

随着主体变得更加复杂和广泛，有效治理的前进道路变得更加重要和必要。

We need practical and multistakeholder processes that promote human safety and security.

我们需要切实可行的多利益相关方流程来促进人类安全。

These types of systems will require continuous monitoring, real-world evaluation, stronger feedback loops, escalation mechanisms, better benchmarks, and coordination among industry, governments, civil society, affected communities, and international frameworks.

这些类型的系统将需要持续监控、现实世界评估、更强大的反馈循环、升级机制、更好的基准以及行业、政府、民间社会、受影响社区和国际框架之间的协调。

A defined shared responsibility across developers, deployers, and users is critical, along with a push to involve affected communities, civil society, regulators, academia, and industry in a co-creation process.

开发人员、部署人员和用户之间明确的共同责任至关重要，同时还要推动受影响社区、民间社会、监管机构、学术界和行业参与共同创造过程。