AI Safety Institute
In plain wordsA government agency that tests the most advanced AI models for dangerous abilities and studies how to make them safe. Britain and the United States set up the first ones in 2023.
A publicly funded technical body that tests advanced AI systems and develops the science of measuring their capabilities and risks. Typical work includes testing frontier modelsfrontier model, n.: One of the most capable AI models in existence at a given time, built by a handful of leading labs at great cost, and the main target of AI safety rules. before and after release under voluntary agreements with developers; dangerous-capability evaluationsdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself. in areas such as cyberattack, chemical and biological weapons and autonomous behavior; research on safeguards and evaluation methods; and advice to governments. The UK and US institutes were announced in November 2023, and Japan, Singapore, South Korea, Canada, France and others followed; the EU takes part through its AI OfficeAI Office, prop. n.: The European Commission's team that oversees the EU AI Act, especially the rules for the largest AI models, and coordinates how the law is applied across Europe.. The institutes are not regulators and generally cannot block a release. Their international network, launched in November 2024, was renamed the International Network for Advanced AI Measurement, Evaluation and Science in December 2025; its members are Australia, Canada, the EU, France, Japan, Kenya, South Korea, Singapore, the United Kingdom and the United States.
The model was tested by both the UK and US institutes before launch.
Origin
The UK institute grew out of the government's Frontier AI Taskforce and was launched at the Bletchley summit in November 2023, alongside the US AI Safety Institute at NIST. In 2025 both were renamed with a shift of emphasis: the UK body became the AI Security Institute in February, and the US institute became the Center for AI Standards and InnovationCenter for AI Standards and Innovation, prop. n.: The US government's AI testing and standards center, part of the National Institute of Standards and Technology. It evaluates advanced AI models, American and foreign, for national-security risks and works with companies on voluntary testing. in June.
Usage note
The 2025 renamings shifted emphasis from broad 'safety' toward national security, standards and measurement science; critics read them as a retreat from wider societal harms, supporters as a sharper focus.
Related
- AI Safety SummitAI Safety Summit, prop. n.: A series of international meetings of governments, AI companies and researchers, begun in England in 2023, to agree on how to handle the risks and benefits of the most advanced AI.
- Center for AI Standards and InnovationCenter for AI Standards and Innovation, prop. n.: The US government's AI testing and standards center, part of the National Institute of Standards and Technology. It evaluates advanced AI models, American and foreign, for national-security risks and works with companies on voluntary testing.
- dangerous capability evaluationdangerous capability evaluation, n.: A test of whether an AI system can do things that could cause severe harm in the wrong hands or on its own — such as helping make a bioweapon, hacking computers or copying itself.
- third-party AI evaluationthird-party AI evaluation, n.: Testing of an AI system by someone other than the company that made it — a government institute, a nonprofit lab or independent researchers — to check what it can do and how risky it is.
- International AI Safety ReportInternational AI Safety Report, prop. n.: A large scientific review, led by the computer scientist Yoshua Bengio and backed by more than 30 countries, summing up what is known about what advanced AI can do and how it could go wrong.
- AI OfficeAI Office, prop. n.: The European Commission's team that oversees the EU AI Act, especially the rules for the largest AI models, and coordinates how the law is applied across Europe.
- frontier AIfrontier AI, n.: The most capable general-purpose AI systems of the moment — those at or beyond the leading edge — which governments single out for extra safety attention.
Sources
- UK Department for Science, Innovation and Technology (2025). Efforts to share best practices on AI measurement and evaluations driven forward through the International Network for Advanced AI Measurement, Evaluation and Science. Press release, 9 December 2025. gov.uk/government/news/efforts-to-share-best-practices-on-ai-measurement-and-evaluations-driven-forward-through-the-international-network-for-advanced-ai-measurement-evalua
Mentioned in
AI safetyAI safety, n.: The effort to keep AI from doing serious harm, whether because it pursues aims its makers never intended, is turned to bad uses or simply malfunctions — a concern that grows as systems become more capable., AI Safety SummitAI Safety Summit, prop. n.: A series of international meetings of governments, AI companies and researchers, begun in England in 2023, to agree on how to handle the risks and benefits of the most advanced AI., AI OfficeAI Office, prop. n.: The European Commission's team that oversees the EU AI Act, especially the rules for the largest AI models, and coordinates how the law is applied across Europe., Bletchley DeclarationBletchley Declaration, prop. n.: A short statement signed by 28 countries and the European Union at the first global AI safety summit, held in England in 2023, agreeing that the most advanced AI could cause serious harm., Center for AI SafetyCenter for AI Safety, prop. n.: A San Francisco nonprofit that studies how advanced AI could lead to disaster and how to prevent it. In 2023 it gathered signatures for a one-sentence warning that AI could threaten human extinction., Center for AI Standards and InnovationCenter for AI Standards and Innovation, prop. n.: The US government's AI testing and standards center, part of the National Institute of Standards and Technology. It evaluates advanced AI models, American and foreign, for national-security risks and works with companies on voluntary testing., Executive Order 14110Executive Order 14110, prop. n.: A sweeping 2023 order by US President Joe Biden that told federal agencies how to manage AI risks and required makers of the largest AI models to report their safety tests to the government. It was revoked in 2025., International AI Safety ReportInternational AI Safety Report, prop. n.: A large scientific review, led by the computer scientist Yoshua Bengio and backed by more than 30 countries, summing up what is known about what advanced AI can do and how it could go wrong., Paul ChristianoPaul Christiano, prop. n.: An American AI-alignment researcher who led the work that made training AI from human preferences practical, the basis of how chatbots are taught to be helpful., pre-deployment testingpre-deployment testing, n.: Checking a new AI model for dangerous abilities and bad behavior before it is released to the public, sometimes with outside experts or government testers given early access., safety casesafety case, n.: A written, evidence-backed argument that a particular AI system is safe enough for a particular use, putting the burden of proof on the developer, as with nuclear plants or aircraft., third-party AI evaluationthird-party AI evaluation, n.: Testing of an AI system by someone other than the company that made it — a government institute, a nonprofit lab or independent researchers — to check what it can do and how risky it is., CybenchCybench, prop. n.: A test of AI agents' hacking skills built from 40 professional 'capture the flag' security puzzles, used to judge how much an AI could help cyberattacks or cyber defense.