How are countries evaluating AI models?

This background briefing guide was distributed to participants ahead of the Forum for Cooperation on AI (FCAI) dialogue on Sept.

本背景简介指南已在 9 月 12 日人工智能合作论坛 (FCAI) 对话之前分发给与会者。

15 , 2026.

2026 年 15 日。

FCAI dialogues follow Chatham House rule, so remarks are anonymized.

FCAI 对话遵循查塔姆研究所规则，因此言论是匿名的。

Artificial intelligence (AI) evaluations aim to measure AI models or systems under specified conditions for a particular capability or risk, or for systemic capabilities, risks, performance, or societal impact.

人工智能（AI）评估旨在衡量特定条件下人工智能模型或系统的特定能力或风险，或系统能力、风险、绩效或社会影响。

Directed evaluations typically select a target and measure an observable proxy for it, aiming to minimize the construct validity gap, or the distance between the target and the proxy.

定向评估通常选择一个目标并为其测量可观察的代理，旨在最小化构建有效性差距或目标与代理之间的距离。

Exploratory evaluations can also be used to anticipate future AI system properties, impacts, risks, use cases , or areas of uncertainty.

探索性评估还可用于预测未来人工智能系统的属性、影响、风险、用例或不确定领域。

Both evaluation types can serve as measurement tools for policymakers and regulators.

这两种评估类型都可以作为决策者和监管者的衡量工具。

Governments and organizations of various kinds are building evaluation capacity simultaneously, and recent events gave that work new urgency.

各国政府和各类组织正在同时建设评估能力，最近发生的事件给这项工作带来了新的紧迫性。

In June, Anthropic released Claude Mythos 5 and Claude Fable 5, models reportedly capable of discovering previously unknown software vulnerabilities and generating working exploits.

6 月，Anthropic 发布了 Claude Mythos 5 和 Claude Fable 5，据报道，这两个模型能够发现以前未知的软件漏洞并生成可利用的漏洞。

Anthropic’s prerelease of these models through Project Glasswing , a collaborative security review process, prompted President Donald Trump to issue an executive order calling for a voluntary prerelease review framework.

Anthropic 通过 Project Glasswing（一个协作安全审查流程）预发布了这些模型，促使唐纳德·特朗普总统发布了一项行政命令，呼吁建立自愿预发布审查框架。

Weeks later, the administration moved to halt foreign nationals’ access to the model through an export-control directive, which was lifted by the end of June.

几周后，政府通过一项出口管制指令，禁止外国人使用该模型，该指令于 6 月底取消。

The following month, OpenAI models broke out of their test environment and hacked the Hugging Face platform to game model tests.

接下来的一个月，OpenAI 模型突破了测试环境，并入侵了 Hugging Face 平台进行模型测试。

In September, Google disclosed that its AI model, Gemini, breached three other companies during a cybersecurity model evaluation conducted by third-party firm Irregular in May.

今年 9 月，谷歌披露其人工智能模型 Gemini 在第三方公司 Irregular 于 5 月进行的网络安全模型评估中违反了其他三家公司的要求。

General-purpose AI (GPAI) models have no defined task, envelope, or intended use to verify against; their utility is extraordinarily broad.

通用人工智能（GPAI）模型没有明确的任务、范围或预期用途来验证；它们的用途非常广泛。

The models are complex, opaque, and unpredictable, and evaluation practices are still forming.

这些模型复杂、不透明且不可预测，评估实践仍在形成中。

Certain capabilities and risks—cybersecurity offense, chemical, biological, radiological, and nuclear (CBRN) uplift, and misinformation—are widely recognized and subject to testing, but new capabilities are emerging faster than methods for evaluating even the consensus risks.

某些能力和风险——网络安全犯罪、化学、生物、放射性和核（CBRN）提升以及错误信息——已被广泛认可并接受测试，但新能力的出现速度甚至比评估共识风险的方法还要快。

Evaluations can target any part of the model supply chain , at different development and deployment stages, or an entire AI system in its deployment context.

评估可以针对处于不同开发和部署阶段的模型供应链的任何部分，或者部署环境中的整个人工智能系统。

Model safety evaluations focus on model outputs and tend to evaluate capabilities.

模型安全评估侧重于模型输出，倾向于评估能力。

By contrast, contextual safety evaluations assess a model as deployed: embedded in a particular product, wrapped in a particular set of safeguards, and used by a particular population.

相比之下，情境安全评估评估部署的模型：嵌入特定产品中，包含在一组特定的防护措施中，并由特定人群使用。

Beyond measuring what a model is capable of in isolation, a sociotechnical approach asks what happens when people use it: whether they can complete their intended tasks, whether outputs fit the context of use, and whether adversarial users can bypass safety restrictions.

除了单独衡量模型的能力之外，社会技术方法还询问人们使用模型时会发生什么：他们是否能够完成预期任务，输出是否适合使用环境，以及敌对用户是否可以绕过安全限制。

This framing measures both whether a behavior can be elicited and how often it arises under realistic conditions of use.

该框架衡量了是否可以引发行为以及在实际使用条件下该行为出现的频率。

Definitions vary across developers, regulators, and adopters.

不同开发者、监管者和采用者的定义各不相同。

Developers and regulators often use “capability evaluation” to determine whether an AI system is safe to deploy, whereas adopters may use the same term for the targeted measurement of positive capabilities.

开发人员和监管机构经常使用“能力评估”来确定人工智能系统是否可以安全部署，而采用者可能会使用相同的术语来有针对性地衡量积极能力。

The EU AI Act references evaluations in various sections, some oriented to capabilities or performance and others to risks (see Appendix A).

欧盟人工智能法案引用了各个部分的评估，一些针对能力或绩效，另一些针对风险（参见附录 A）。

Benchmark evaluations now report that GPAI systems perform at or above human-expert baselines on several academic question sets, including scores over 90% on undergraduate-level exams in subjects from chemistry to law (MMLU) and over 80% on graduate-level science questions (GPQA).

基准评估现在报告显示，GPAI 系统在多个学术问题集上的表现达到或高于人类专家基线，包括化学到法律等科目的本科水平考试 (MMLU) 得分超过 90%，研究生水平科学问题 (GPQA) 得分超过 80%。

But exceeding a human baseline on a fixed set of questions does not indicate expert-level competence, and performance degrades quickly as tasks lengthen.

但在一组固定问题上超过人类基线并不表示专家级的能力，并且随着任务的延长，表现会迅速下降。

Model Evaluation and Threat Research (METR) measures a model’s “time horizon,” the length of task, measured by how long human experts take to complete it, at which a model succeeds at a given rate of reliability.

模型评估和威胁研究 (METR) 衡量模型的“时间范围”，即任务的长度，以人类专家完成任务所需的时间来衡量，在此时间范围内模型以给定的可靠性率取得成功。

Leading models reach 50% reliability on software tasks that take human experts roughly an hour and fall below 10% on tasks that take humans more than about four hours.

对于人类专家需要大约一个小时的软件任务，领先模型的可靠性达到 50%；而对于人类需要大约四个小时以上的任务，领先模型的可靠性则低于 10%。

No evaluation can give the full picture.

任何评估都无法提供全面的信息。

What an evaluation leaves out is itself a choice, as are its goal, scope, and interpretation of results.

评估遗漏的本身就是一种选择，评估的目标、范围和结果解释也是如此。

These choices reflect the perspective and motivation of its designers.

这些选择反映了其设计者的观点和动机。

Evaluations are therefore not neutral instruments : Deciding what to measure, how to measure it, and how to interpret results requires value judgment from evaluators.

因此，评估不是中立的工具：决定测量什么、如何测量以及如何解释结果需要评估者的价值判断。

Evaluation methods for AI did not begin with current frontier models.

人工智能的评估方法并不是从当前的前沿模型开始的。

They build upon software testing methods but require larger scale and scope.

它们建立在软件测试方法的基础上，但需要更大的规模和范围。

Predictive machine learning systems have long been evaluated against a known task distribution .

预测机器学习系统长期以来一直根据已知的任务分布进行评估。

For example, a model trained to classify loan applications or flag tumors has a defined input, a validation set drawn from the same distribution as its training data, and ground-truth labels.

例如，经过训练对贷款申请进行分类或标记肿瘤的模型具有定义的输入、从与其训练数据相同的分布中提取的验证集以及真实标签。

Even under those conditions, held-out performance proved an unreliable guide to deployment behavior .

即使在这些条件下，保留的性能也被证明是部署行为的不可靠指南。

Foundation models break the assumption of a defined task distribution.

基础模型打破了定义任务分布的假设。

Since a GPAI model is trained on broad data and adapted to a wide range of downstream tasks, no held-out set represents deployment.

由于 GPAI 模型是在广泛的数据上进行训练并适应广泛的下游任务，因此没有保留的集合代表部署。

Benchmarks therefore serve as proxies for capabilities that cannot be directly measured (the construct validity challenge).

因此，基准可以作为无法直接测量的能力的代理（构造有效性挑战）。

Measured capability also depends on how a model is prompted, what tools it is given, and how much compute it is allowed.

衡量的能力还取决于模型的提示方式、提供的工具以及允许的计算量。

An evaluation that does not attempt to elicit a model’s full capability shows a floor of what the model did under those particular conditions, not what it may be able to do .

不试图得出模型全部能力的评估显示的是模型在这些特定条件下所做的事情的底线，而不是它可能能够做的事情。

A later evaluator with better prompting may be able to find capabilities in the same model that the first one missed.

后来的评估者如果有更好的提示，也许能够在同一模型中找到第一个评估者错过的功能。

Agentic systems shift the unit of evaluation again: An agent completes multistep trajectories, has tool-calling capabilities, and acts on an environment .

代理系统再次改变评估单位：代理完成多步轨迹，具有工具调用能力，并对环境起作用。

Evaluations must assess the deployed AI system: the model and its tools, permissions, memory, and operating environment.

评估必须评估已部署的人工智能系统：模型及其工具、权限、内存和操作环境。

The National Institute of Standards and Technology’s (NIST) TEVV-Athlon draft extends to agentic systems on this basis, and Singapore’s AI Safety Institute (AISI) frames the shift as testing systems rather than models .

美国国家标准与技术研究所 (NIST) 的 TEVV-Athlon 草案在此基础上扩展到代理系统，新加坡人工智能安全研究所 (AISI) 将这种转变定义为测试系统而不是模型。

Capability evaluations conducted in a testing environment can miss contextual factors of human-AI interaction in deployment settings.

在测试环境中进行的能力评估可能会错过部署环境中人机交互的上下文因素。

An evaluation may measure the likelihood of a system producing factually incorrect output, but the actual risk of people being deceived depends on the context of use , the users, and features of the application, such as whether AI-generated content is watermarked.

评估可以衡量系统产生事实上不正确的输出的可能性，但人们被欺骗的实际风险取决于使用环境、用户和应用程序的功能，例如人工智能生成的内容是否带有水印。

Methods for measuring risk in context exist.

存在衡量上下文风险的方法。

At the human interaction level, risk is typically measured through user studies and uplift studies comparing task outcomes with and without model access under test and control conditions.

在人机交互层面，风险通常通过用户研究和提升研究来衡量，比较测试和控制条件下有和没有模型访问的任务结果。

NIST’s Assessing Risks and Impacts of AI (ARIA) program uses three tiers: (1) model testing to assess whether a system performs as advertised, (2) red teaming to probe adversarial use, and (3) field testing to evaluate behavior in realistic scenarios with human users.

NIST 的人工智能风险和影响评估 (ARIA) 计划使用三个层次：(1) 模型测试来评估系统是否按照宣传的那样运行；(2) 红队来调查对抗性使用；(3) 现场测试来评估人类用户在现实场景中的行为。

ARIA field testing may involve several thousand participants interacting with AI applications across multiple sessions under test or control conditions, scored against a Contextual Robustness Index (CoRIx), which gives a suite of metrics to measure AI systems’ functionality across different deployment conditions.

ARIA 现场测试可能涉及数千名参与者在测试或控制条件下跨多个会话与 AI 应用程序进行交互，并根据上下文稳健性指数 (CoRIx) 进行评分，该指数提供了一套指标来衡量 AI 系统在不同部署条件下的功能。

Singapore’s Global AI Assurance Pilot applied similar logic at the application layer by pairing assurance vendors with firms deploying live generative AI products.

新加坡的全球人工智能保障试点通过将保障供应商与部署实时生成人工智能产品的公司配对，在应用层应用了类似的逻辑。

At the systemic layer, methods are thinner, relying on post-market monitoring, incident reporting, and observational study rather than controlled testing—one reason the EU Code of Practice ties escalation of assessment depth to post-market and incident data rather than to prerelease testing alone.

在系统层面，方法更薄弱，依赖于上市后监测、事件报告和观察性研究，而不是受控测试——这就是欧盟实践守则将评估深度升级与上市后和事件数据联系起来，而不是仅仅与发布前测试联系起来的原因。

Capabilities may contribute to large-scale effects, but a capability evaluation cannot determine outcomes without examining systemic factors .

能力可能会产生大规模的影响，但如果不考察系统性因素，能力评估就无法确定结果。

AI-generated misinformation likely affects public trust in shared media, for instance, but capability metrics alone will not illustrate a trend without analysis of norms in information sharing or the existence of fact-checking institutions.

例如，人工智能生成的错误信息可能会影响公众对共享媒体的信任，但如果不分析信息共享规范或事实核查机构的存在，仅靠能力指标并不能说明趋势。

Capability evaluation gives an early indication of potential downstream harms but requires context to determine risks of harm.

能力评估可以尽早表明潜在的下游危害，但需要背景信息来确定危害风险。

Evaluators across both public and private sectors have differing relationships with developers and deployers, which affects their access to information and their objectivity and independence.

公共和私营部门的评估人员与开发人员和部署人员有着不同的关系，这影响了他们对信息的获取以及客观性和独立性。

There have been proposed scales for types of evaluations by level of independence (Figure A).

已经提出了按独立程度划分的评估类型的量表（图 A）。

First-party evaluations are run in-house on an organization’s own models, data, or applications.

第一方评估是在组织自己的模型、数据或应用程序内部运行的。

The authors distinguish first-party limited, carried out by the team responsible for building and validating the system’s performance, from first-party expansive, carried out by a separate team dedicated to flaw detection but not responsible for building the system, such as a red team or internal compliance team.

作者区分了第一方有限（由负责构建和验证系统性能的团队执行）和第一方扩展（由专门用于缺陷检测但不负责构建系统的单独团队（例如红队或内部合规团队）执行）。

Second-party evaluators have a contractual or obligatory relationship with the provider or purchaser.

第二方评估者与提供者或购买者具有合同或义务关系。

This could include a developer providing a customer with model access or a contracted organization evaluating on a developer’s behalf.

这可能包括开发人员向客户提供模型访问权限或代表开发人员进行评估的签约组织。

Their work is usually more formal because the contractual terms lay out the evaluation.

他们的工作通常更加正式，因为合同条款规定了评估。

Second-party limited evaluations are carried out by a specific contracted party and are limited in time and scope; second-party expansive evaluations are carried out by various contracted parties.

第二方有限评估由特定承包商进行，有时间和范围限制；第二方广泛评估由各缔约方进行。

Paid evaluation consultancies and potential purchasers of an AI system are other examples.

其他例子还有付费评估咨询公司和人工智能系统的潜在购买者。

Third-party evaluators have no existing contractual relationship with the developers or deployers.

第三方评估者与开发者或部署者没有现有的合同关系。

The paper distinguishes between preapproved third-party evaluations, where the provider vets evaluators and provides access beforehand; third-party limited evaluations, which are carried out without safe harbor; and third-party expansive evaluations, where third parties have safe harbor and coordinated flaw-reporting infrastructure.

该论文区分了预先批准的第三方评估，即提供商预先审查评估者并提供访问权限；第三方有限评估，在没有安全港的情况下进行；第三方扩展评估，第三方拥有安全港和协调的缺陷报告基础设施。

Third-party evaluations can also be categorized by who decides to conduct them.

第三方评估也可以根据决定进行评估的人进行分类。

They can be evaluatee-driven, where the company being evaluated hires the evaluator and may set a specific objective, ask for evaluations certifying compliance with an independent standard, or grant a wide remit to conduct exploratory evaluations; government-driven, where a government or independent regulator appoints an evaluator or evaluates directly; or independent, where academics, journalists, or civil society organizations evaluate based on their own priorities, with or without the developer’s oversight.

它们可以是评估者驱动的，被评估的公司聘请评估员，并可能设定具体目标，要求评估证明符合独立标准，或授予广泛的职权范围以进行探索性评估；政府驱动，由政府或独立监管机构指定评估员或直接评估；或独立，学者、记者或民间社会组织根据自己的优先事项进行评估，无论有或没有开发商的监督。

Some jurisdictions require risk-assessment evaluations for AI systems in certain contexts.

一些司法管辖区要求在某些情况下对人工智能系统进行风险评估。

For example, New York City’s Local Law 144 requires annual bias audits by an independent auditor for automated employment-decision tools.

例如，纽约市第 144 号地方法要求独立审计员对自动化就业决策工具进行年度偏差审计。

Different evaluations require varying levels of system access, from public queries, probabilities, gradients, weights and activations, and fine-tuning access to outside-the-box and unrestricted access.

不同的评估需要不同级别的系统访问，从公共查询、概率、梯度、权重和激活，以及微调访问到开箱即用和无限制访问。

AI evaluation designers make decisions about what to measure, how to measure, and how to interpret their results.

人工智能评估设计者决定测量什么、如何测量以及如何解释结果。

NIST’s Center for AI Standards and Innovation’s (CAISI) initial public draft of NIST AI 800-2, “ Practices for Automated Benchmark Evaluations of Language Models ” recommends that designers ask two questions: How will the measurements be used, and what should be measured?

NIST 人工智能标准与创新中心 (CAISI) 的 NIST AI 800-2 初始公开草案“语言模型自动基准评估实践”建议设计者提出两个问题：如何使用测量以及应该测量什么？

The answer to the first question informs the second, and designers then determine and record the measurement construct for selected use cases.

第一个问题的答案告知第二个问题，然后设计人员确定并记录所选用例的测量构造。

Potential uses of evaluation results include informing or assessing development decisions; assessing whether a system is fit to use for a specific scenario; comparing systems for deployment readiness; assessing deployment configuration efficacy; validating that a system has been deployed and configured correctly; and informing predictions of real-world impacts.

评估结果的潜在用途包括告知或评估开发决策；评估系统是否适合特定场景；比较系统的部署准备情况；评估部署配置的有效性；验证系统是否已正确部署和配置；并提供对现实世界影响的预测。

Depending on the objective, an evaluation may measure a model, a system, or an individual component or algorithm.

根据目标，评估可以衡量模型、系统或单个组件或算法。

Table 1 presents examples of goals and associated measurement constructs.

表 1 列出了目标和相关测量结构的示例。

NIST released the initial public draft of AI 200-2, the TEVV-Athlon Framework for Evaluating AI Systems, on Aug.

NIST 于 2018 年 8 月发布了 AI 200-2（用于评估人工智能系统的 TEVV-Athlon 框架）的初始公开草案。

7, with comments due Oct.

7，评论截止日期为 10 月

6.

6.

Where AI 800-2 addresses automated benchmark evaluations end to end, from defining objectives through analyzing and reporting results, AI 200-2 is a four-stage method for building customized assessments from an organization’s TEVV objectives rather than a standard list of system tests.

AI 800-2 解决端到端的自动化基准评估，从定义目标到分析和报告结果，而 AI 200-2 是一种四阶段方法，用于根据组织的 TEVV 目标而不是标准的系统测试列表构建定制评估。

NIST describes it as extensible across statistical machine learning models, LLMs, multimodal models, and agentic systems.

NIST 将其描述为可跨统计机器学习模型、法学硕士、多模式模型和代理系统扩展。

Model safety evaluations, which tend to measure capabilities, are often conducted through capability testing or benchmarking (shared tests used to compare how well AI systems perform specific tasks).

模型安全评估倾向于衡量能力，通常通过能力测试或基准测试（用于比较人工智能系统执行特定任务的情况的共享测试）来进行。

Contextual safety evaluations, which assess a model as deployed, with its safeguards and users, may include red-teaming (a structured testing effort, often adopting adversarial methods, to find flaws and vulnerabilities in an AI system) or uplift studies (comparison of how testers complete a task with and without model access to evaluate whether the model provided meaningful assistance).

上下文安全评估评估已部署的模型及其保障措施和用户，可能包括红队（一种结构化测试工作，通常采用对抗性方法，以发现人工智能系统中的缺陷和漏洞）或提升研究（比较测试人员在有和没有模型访问的情况下如何完成任务，以评估模型是否提供了有意义的帮助）。

Red-teaming often relies on a manual human-led process, which takes longer but allows teams to adapt evaluations to the specific models and systems at hand.

红队通常依赖于人工主导的手动流程，该流程需要更长的时间，但允许团队根据手头的特定模型和系统调整评估。

Some benchmark evaluations run automated , without a human in the loop.

一些基准评估是自动运行的，无需人工参与。

Human oversight generally increases accuracy, but automated benchmarks may be less costly , quicker, and easier to scale.

人工监督通常会提高准确性，但自动化基准可能成本更低、速度更快且更容易扩展。

Since benchmarks are built by many parties under no common methodological standard, they vary significantly in quality, scope, validity, and robustness.

由于基准是由多方在没有共同方法标准的情况下制定的，因此它们在质量、范围、有效性和稳健性方面存在很大差异。

Some researchers have argued for better deprecation practices for benchmarks, given the static nature of the evaluations.

考虑到评估的静态性质，一些研究人员主张对基准采取更好的弃用做法。

Some frontier model developers have also used automated red-teaming .

一些前沿模型开发人员也使用了自动化红队。

Automated evaluations have been criticized as “superficial” and “unreliable,” and for overestimating model utility.

自动评估被批评为“肤浅”和“不可靠”，并且高估了模型效用。

Evaluations of harmful outcomes test different outcomes than capability tests.

有害结果的评估测试的结果与能力测试不同。

Typically, evaluators will run prompts from a harm taxonomy against the model and score whether the model complied, and if the result was actionable and could result in a harmful outcome.

通常，评估人员会针对模型运行危害分类的提示，并对模型是否符合要求以及结果是否可行并可能导致有害结果进行评分。

Scores do not always reflect whether it was a single-turn prompt or a larger number of attempts that resulted in actionable outputs, or if prompting changed over time.

分数并不总是反映是单轮提示还是大量尝试产生了可操作的输出，或者提示是否随着时间的推移而改变。

When CAISI red-teamed an agent in collaboration with the U.K. AI Security Institute using attacks developed specifically for that model rather than known baselines, the measured attack success rate rose from 11% to 81%.

当 CAISI 与英国人工智能安全研究所合作，使用专门为该模型开发的攻击而不是已知基线来对代理进行红队时，测得的攻击成功率从 11% 上升到 81%。

Evaluation may also include model elicitation , or “technical work to systematically enhance a model’s capabilities, propensities, affordances, and/or effects, thereby facilitating an accurate measurement of the full range of its capabilities, propensities, affordances, and/or effects that can likely be attained” to test a fuller extent of model capabilities.

评估还可能包括模型启发，或“系统地增强模型的能力、倾向、可供性和/或效果的技术工作，从而促进准确测量其可能达到的全部能力、倾向、可供性和/或效果”，以测试更全面的模型能力。

New York City’s Local Law 144 and the EU AI Act both use disaggregated evaluation methods to measure bias and disparate performance.

纽约市第 144 号地方法和欧盟人工智能法案均使用分类评估方法来衡量偏见和差异表现。

Known limits to this evaluation approach include that subgroups must be specified in advance and that demographic information needed for disaggregation can be unavailable or legally restricted .

这种评估方法的已知限制包括必须提前指定子组，以及分解所需的人口统计信息可能无法获得或受到法律限制。