AI governance
In plain wordsThe rules, institutions and habits that shape how AI is built and used — from laws and treaties to technical standards and a company's own review processes.
- 1
Public policy
The norms, institutions and processes through which states, international bodies, companies and civil society steer the development and use of artificial intelligenceartificial intelligence, n.: Computer systems that do things we would normally credit to a thinking mind, such as holding a conversation, reading an X-ray or planning a route, and the field of research that builds them. and assign responsibility for its effects. It operates in layers: binding law (AI regulationAI regulation, n.: Laws and binding government rules about artificial intelligence: what AI may be used for, what checks its makers must run, and what penalties apply when they break the rules. such as the EU AI ActEU AI Act, prop. n.: The European Union's law on artificial intelligence, the first broad AI law anywhere: it bans some uses, sets strict checks for risky ones and puts duties on the makers of the biggest AI models.); international instruments (OECD AI PrinciplesOECD AI Principles, prop. n.: Guidelines for trustworthy AI that governments agreed through the OECD in 2019, the first such standard agreed between governments; many later laws and agreements borrowed from them., the Framework Convention on Artificial IntelligenceFramework Convention on Artificial Intelligence, prop. n.: The first binding international treaty on AI, drawn up by the Council of Europe, in which countries promise that AI used by governments and businesses will respect human rights, democracy and the rule of law.); technical standards (ISO/IEC 42001ISO/IEC 42001, prop. n.: An international standard for how an organization should run its AI work responsibly — policies, risk checks, roles and records — which outside auditors can certify, much as ISO 9001 does for quality.); voluntary commitments and industry frameworks (responsible scaling policiesresponsible scaling policy, n.: A public rulebook in which an AI company promises to test each new model for dangerous abilities and to add stronger safeguards, or slow down, as its models become more capable., the Frontier AI Safety CommitmentsFrontier AI Safety Commitments, prop. n.: Promises made in 2024 by major AI companies to publish safety plans for their most advanced models, including the danger points at which they would stop if the risks could not be controlled.); and scientific assessment bodies such as the Independent International Scientific Panel on AIIndependent International Scientific Panel on AI, prop. n.: A group of 40 independent experts set up by the United Nations in 2025 to give every country a shared, science-based picture of what AI can do, its risks and its benefits.. Analysts distinguish hard law from soft law and national from global governance; compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI. targets the chips and data centers needed to train frontier modelsfrontier model, n.: One of the most capable AI models in existence at a given time, built by a handful of leading labs at great cost, and the main target of AI safety rules..
Most of the AI governance debate at the summit was about who should evaluate frontier models before release.
- 2
Organizations
The internal policies, roles and controls by which an organization manages the AI systems it builds, buys or deploys: an inventory of systems, risk tiers, approval gates before deployment, documentation, monitoring, incident handling and board-level accountability. Such programs are commonly structured around the NIST AI Risk Management FrameworkNIST AI Risk Management Framework, prop. n.: A free, voluntary guide from the US government's standards agency that helps organizations spot, measure and reduce the risks of the AI systems they build or use. or certified against ISO/IEC 42001ISO/IEC 42001, prop. n.: An international standard for how an organization should run its AI work responsibly — policies, risk checks, roles and records — which outside auditors can certify, much as ISO 9001 does for quality..
Our AI governance program assigns every deployed model an owner, a risk tier and a review date.
Origin
Built on the older vocabularies of corporate and internet governance. Early academic uses include Urs Gasser and Virgilio Almeida's 'A Layered Model for AI Governance' (2017) and Allan Dafoe's 'AI Governance: A Research Agenda' (Future of Humanity Institute, Oxford, 2018). The phrase became standard policy vocabulary after the 2019 OECD Principles and the 2023 wave of summits and legislation.
Usage note
Broader than AI regulationAI regulation, n.: Laws and binding government rules about artificial intelligence: what AI may be used for, what checks its makers must run, and what penalties apply when they break the rules., which covers only binding public rules. Overlaps with AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect., which supplies values, and AI safetyAI safety, n.: The effort to keep AI from doing serious harm, whether because it pursues aims its makers never intended, is turned to bad uses or simply malfunctions — a concern that grows as systems become more capable., which supplies technical methods.
Related
- AI regulationAI regulation, n.: Laws and binding government rules about artificial intelligence: what AI may be used for, what checks its makers must run, and what penalties apply when they break the rules.
- AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect.
- AI safetyAI safety, n.: The effort to keep AI from doing serious harm, whether because it pursues aims its makers never intended, is turned to bad uses or simply malfunctions — a concern that grows as systems become more capable.
- compute governancecompute governance, n.: Using control over the computer chips and data centers that AI depends on — who can buy them, where they go, how they are used — as a way to oversee powerful AI.
- risk-based approachrisk-based approach, n.: A way of regulating AI in which the strictness of the rules depends on how much harm a use could cause: bans for the worst uses, heavy checks for risky ones, little or nothing for the rest.
- AI Safety SummitAI Safety Summit, prop. n.: A series of international meetings of governments, AI companies and researchers, begun in England in 2023, to agree on how to handle the risks and benefits of the most advanced AI.
- algorithmic accountabilityalgorithmic accountability, n.: The idea that the people and organizations behind automated decision systems must be able to explain and justify those systems' effects, answer for the harm they cause, and fix it.
- Collingridge dilemmaCollingridge dilemma, n.: The trap that a new technology is easiest to control when little is known about its harms, but by the time the harms are clear it is too widespread to change easily.
- AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect.
Sources
- Gasser, U. & Almeida, V. A. F. (2017). A Layered Model for AI Governance. IEEE Internet Computing, 21(6), 58–62.
- Dafoe, A. (2018). AI Governance: A Research Agenda. Centre for the Governance of AI, Future of Humanity Institute, University of Oxford.
Mentioned in
AI ethicsAI ethics, n.: The study of what is right and wrong in how AI is built and used: who it may harm, who is responsible when it fails, and which values it should respect., AI regulationAI regulation, n.: Laws and binding government rules about artificial intelligence: what AI may be used for, what checks its makers must run, and what penalties apply when they break the rules., AI riskAI risk, n.: The ways AI could cause harm: people misusing it on purpose, systems behaving in ways nobody intended, and slower effects on jobs, power and society that no single person causes., AI Safety SummitAI Safety Summit, prop. n.: A series of international meetings of governments, AI companies and researchers, begun in England in 2023, to agree on how to handle the risks and benefits of the most advanced AI., Brussels effectBrussels effect, n.: The way European Union rules end up shaping products and laws worldwide, because global companies often find it simpler to follow the strictest big market's rules everywhere., Cyberspace Administration of ChinaCyberspace Administration of China, prop. n.: China's top internet authority, in charge of online censorship as well as regulation, which also writes and enforces most of the country's AI rules, such as the registration of chatbots and recommendation algorithms., ethics washingethics washing, n.: Using talk of ethics, such as principles, advisory boards or pledges, to look responsible and head off regulation, without actually changing what a company does., foundation modelfoundation model, n.: A very large AI model, built once at great cost from vast amounts of text, images or code gathered mostly from the internet, that many products then reuse as their starting point, from chatbots to coding tools., Framework Convention on Artificial IntelligenceFramework Convention on Artificial Intelligence, prop. n.: The first binding international treaty on AI, drawn up by the Council of Europe, in which countries promise that AI used by governments and businesses will respect human rights, democracy and the rule of law., G7 Hiroshima AI ProcessG7 Hiroshima AI Process, prop. n.: An effort begun by the G7 group of wealthy democracies in Japan in 2023 that sets voluntary rules for companies building the most advanced AI, which now report publicly on how they follow them., Global Digital CompactGlobal Digital Compact, prop. n.: A 2024 agreement among United Nations member countries setting shared goals for the digital world — closing digital divides, protecting rights online, governing data and AI — and creating new UN bodies on AI., Independent International Scientific Panel on AIIndependent International Scientific Panel on AI, prop. n.: A group of 40 independent experts set up by the United Nations in 2025 to give every country a shared, science-based picture of what AI can do, its risks and its benefits., international humanitarian lawinternational humanitarian law, n.: The rules of international law that limit how wars may be fought, protecting civilians, the wounded and prisoners. They apply to every weapon, including ones that use AI., ISO/IEC 42001ISO/IEC 42001, prop. n.: An international standard for how an organization should run its AI work responsibly — policies, risk checks, roles and records — which outside auditors can certify, much as ISO 9001 does for quality., model risk managementmodel risk management, n.: The rules banks and other firms follow to make sure the mathematical models and AI systems they rely on are built correctly, checked independently and used only for what they are fit for., NIST AI Risk Management FrameworkNIST AI Risk Management Framework, prop. n.: A free, voluntary guide from the US government's standards agency that helps organizations spot, measure and reduce the risks of the AI systems they build or use., OECD AI PrinciplesOECD AI Principles, prop. n.: Guidelines for trustworthy AI that governments agreed through the OECD in 2019, the first such standard agreed between governments; many later laws and agreements borrowed from them., OpenAI leadership crisis (2023)OpenAI leadership crisis (2023), n.: The five days in November 2023 when OpenAI's board fired chief executive Sam Altman, nearly all staff threatened to quit, and he returned with a new board., responsible AIresponsible AI, n.: An umbrella name for the practices organizations use to keep their AI systems fair, safe, private and accountable, from the first design meeting to everyday use., UNESCO Recommendation on the Ethics of AIUNESCO Recommendation on the Ethics of AI, prop. n.: A worldwide set of ethical principles for AI that all of UNESCO's member countries agreed in 2021, covering human rights, fairness, privacy, the environment, gender equality and education., Agentic AI FoundationAgentic AI Foundation, prop. n.: A nonprofit group under the Linux Foundation that looks after shared, open technical standards for AI agents, such as the Model Context Protocol, so that no single company controls them., AI PactAI Pact, prop. n.: A voluntary European Commission scheme in which companies pledge to start following the EU AI Act's main ideas early, before the rules legally apply to them., AI Promotion ActAI Promotion Act, prop. n.: Japan's 2025 national AI law, which sets up a government AI headquarters and strategy to encourage AI development and use, relying on cooperation and guidance rather than fines., Center for Security and Emerging TechnologyCenter for Security and Emerging Technology, prop. n.: A Georgetown University think tank in Washington, D.C., whose researchers advise the US government on AI and national security, paying special attention to China and to the chip supply chain.