third-party AI evaluation
In plain wordsTesting of an AI system by someone other than the company that made it — a government institute, a nonprofit lab or independent researchers — to check what it can do and how risky it is.
Assessment of an AI model's capabilities, risks or safeguards by parties independent of its developer: government bodies such as the UK AI Security InstituteAI Safety Institute, n.: A government agency that tests the most advanced AI models for dangerous abilities and studies how to make them safe. Britain and the United States set up the first ones in 2023. and the US Center for AI Standards and InnovationCenter for AI Standards and Innovation, prop. n.: The US government's AI testing and standards center, part of the National Institute of Standards and Technology. It evaluates advanced AI models, American and foreign, for national-security risks and works with companies on voluntary testing., specialist nonprofits such as METRMETR, prop. n.: A nonprofit research group that checks new AI systems for dangerous skills, best known for tracking how long the tasks are that AI agents can now finish on their own. and Apollo ResearchApollo Research, prop. n.: An AI safety organization, founded in London in 2023, that tests advanced AI systems for signs that they might secretly pursue goals their developers did not intend., contracted firms and academic researchers. It ranges from pre-deployment testingpre-deployment testing, n.: Checking a new AI model for dangerous abilities and bad behavior before it is released to the public, sometimes with outside experts or government testers given early access. under access agreements, through red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them. and bug bounties, to research on released models. Its value depends on the depth of access (black-box API queries, fine-tuning access, chains of thought, or white-box access to weights and activations), the time allowed before release, and the evaluator's independence and freedom to publish. Researchers have called for legal and contractual safe harbors so that good-faith testing does not breach terms of service. The EU General-Purpose AI Code of PracticeGeneral-Purpose AI Code of Practice, prop. n.: A voluntary rulebook, published by the EU in 2025, that makers of large AI models can sign to show they follow the EU AI Act's rules on transparency, copyright and safety. expects external evaluation of systemic-risk models, and California's SB 53SB 53, prop. n.: A 2025 California law requiring developers of the most powerful AI models to publish their safety plans, report serious safety incidents to the state and protect employees who raise alarms. requires large developers to describe their use of third-party assessors.
The system card lists two third-party evaluations, each with only ten days of pre-release access.
Origin
External pre-release testing of frontier models began with ARC Evals' (now METR) evaluation of GPT-4 before its March 2023 launch; government institutes began pre-deployment testing in 2024, the US institute under agreements signed with OpenAI and Anthropic that August. The March 2024 open letter 'A Safe Harbor for AI Evaluation and Red Teaming' (Shayne Longpre and colleagues) pressed for legal protection.
Related
- AI auditAI audit, n.: An independent check of an AI system, or of the company using it, against agreed standards — for example, testing whether a hiring tool treats women and men fairly.
- pre-deployment testingpre-deployment testing, n.: Checking a new AI model for dangerous abilities and bad behavior before it is released to the public, sometimes with outside experts or government testers given early access.
- dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
- red teamingred teaming, n.: Deliberately attacking your own AI system — trying to make it misbehave, leak secrets or help with harm — so that weaknesses are found before real adversaries or users find them.
- Center for AI Standards and InnovationCenter for AI Standards and Innovation, prop. n.: The US government's AI testing and standards center, part of the National Institute of Standards and Technology. It evaluates advanced AI models, American and foreign, for national-security risks and works with companies on voluntary testing.
- METRMETR, prop. n.: A nonprofit research group that checks new AI systems for dangerous skills, best known for tracking how long the tasks are that AI agents can now finish on their own.
- system cardsystem card, n.: A public report released with an AI model that explains what it can do, how it was tested for safety, what risks were found and what protections were put in place.
Sources
- Longpre, S. et al. (2024). A Safe Harbor for AI Evaluation and Red Teaming. arXiv:2403.04893.
Mentioned in
AI Safety InstituteAI Safety Institute, n.: A government agency that tests the most advanced AI models for dangerous abilities and studies how to make them safe. Britain and the United States set up the first ones in 2023., evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews., AI auditAI audit, n.: An independent check of an AI system, or of the company using it, against agreed standards — for example, testing whether a hiring tool treats women and men fairly., Center for AI Standards and InnovationCenter for AI Standards and Innovation, prop. n.: The US government's AI testing and standards center, part of the National Institute of Standards and Technology. It evaluates advanced AI models, American and foreign, for national-security risks and works with companies on voluntary testing., conformity assessmentconformity assessment, n.: The check, done by the maker or by an approved outside body, that a high-risk AI system meets the EU's legal requirements before it can be sold or used., METRMETR, prop. n.: A nonprofit research group that checks new AI systems for dangerous skills, best known for tracking how long the tasks are that AI agents can now finish on their own., pre-deployment testingpre-deployment testing, n.: Checking a new AI model for dangerous abilities and bad behavior before it is released to the public, sometimes with outside experts or government testers given early access., Centre for the Governance of AICentre for the Governance of AI, prop. n.: A research organization, based in Britain, that studies how governments and companies should handle the dangers of advanced AI and the large economic and political changes it could cause.