AlphaZero

Origin

Described by David SilverDavid Silver, prop. n.: A British computer scientist who led the reinforcement-learning research behind AlphaGo and AlphaZero, programs that taught themselves to play board games better than any human. and colleagues in a December 2017 preprint, 'Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm', and in Science on 7 December 2018.

Usage note

The 2017 matches drew criticism for giving Stockfish restricted hardware and time controls; the longer matches reported in 2018 addressed many of the objections.

Related

Sources

1. Silver, D. et al. (2018). A General Reinforcement Learning Algorithm that Masters Chess, Shogi, and Go through Self-Play. Science 362(6419), 1140–1144.

Mentioned in

AlphaGoAlphaGo, prop. n.: A DeepMind program that in 2016 beat Lee Sedol, one of the world's best players of the board game Go, a decade or more before many experts expected a computer to manage it., Demis HassabisDemis Hassabis, prop. n.: A British AI researcher and former chess prodigy who co-founded DeepMind, led the teams behind AlphaGo and AlphaFold, and shared the 2024 Nobel Prize in Chemistry., AlphaGo ZeroAlphaGo Zero, prop. n.: A program from DeepMind that taught itself to play the board game Go better than any human in 2017, starting from nothing but the rules and learning only by playing millions of games against itself., David SilverDavid Silver, prop. n.: A British computer scientist who led the reinforcement-learning research behind AlphaGo and AlphaZero, programs that taught themselves to play board games better than any human., Deep BlueDeep Blue, prop. n.: IBM's chess computer that beat world champion Garry Kasparov in 1997, the first time a machine defeated a reigning world champion in a match under standard tournament conditions., deep reinforcement learningdeep reinforcement learning, n.: Reinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images., DeepMindDeepMind, prop. n.: A London artificial intelligence company, bought by Google in 2014, famous for programs that mastered the game of Go and predicted the shapes of proteins. In 2023 it merged into Google DeepMind., game theorygame theory, n.: The mathematics of strategic decisions, where each player's best move depends on what the others do. It is used to study competition and cooperation among people, firms, nations and AI agents., Garry KasparovGarry Kasparov, prop. n.: A Russian chess world champion whose 1997 loss to IBM's Deep Blue computer became the symbol of machines overtaking people, and who later championed humans and computers working together., Monte Carlo tree searchMonte Carlo tree search, n.: A way for a game-playing program to choose a move by simulating many possible continuations, focusing on the most promising lines, and picking the move that worked out best on average., Move 37Move 37, prop. n.: A surprising move played by the AI program AlphaGo against champion Lee Sedol in 2016. Experts first thought it a mistake; it proved brilliant, and the phrase now stands for a creative, unhuman idea from AI., self-playself-play, n.: Training an AI by having it play against copies of itself, so it always has an opponent at its own level and can keep improving without human examples., AlphaProofAlphaProof, prop. n.: A Google DeepMind system that learned to write computer-checked mathematical proofs by trial and error. In 2024 it helped an AI reach silver-medal level at the International Mathematical Olympiad for the first time., AlphaStarAlphaStar, prop. n.: A DeepMind AI that learned to play the real-time strategy game StarCraft II well enough to reach its highest public rank, Grandmaster, in 2019., ChinookChinook, prop. n.: A checkers program from the University of Alberta that in 1994 became the first program crowned world champion in a contest against people, and in 2007 proved that perfect checkers always ends in a draw., counterfactual regret minimizationcounterfactual regret minimization, n.: A method for working out near-unbeatable strategies in games with hidden information, like poker, by repeatedly playing against itself and shifting each decision toward the moves it most regrets not making., era of experienceera of experience, n.: The prediction, made by David Silver and Richard Sutton in 2025, that AI will soon learn mainly from its own experience of acting in the world rather than from human-written data., evaluation functionevaluation function, n.: In a game-playing program, a formula that looks at a board position and gives it a score saying how good it is, without playing the game out to the end., expert iterationexpert iteration, n.: A learning loop in which a slow, careful search finds better moves or solutions than the AI's quick instincts, and the AI is then trained to imitate them, so each round's search starts smarter., general game playinggeneral game playing, n.: Building AI players that are handed the rules of a game they have never seen and must work out how to play it well on their own, with no game-specific help from their programmers., MuZeroMuZero, prop. n.: A DeepMind game-playing program from 2019 that mastered Go, chess, shogi and Atari games without being told the rules, by learning its own simplified model of each game to plan with., policy iterationpolicy iteration, n.: A method for finding the best strategy in a fully known problem by alternating two steps: work out how good the current strategy is, then switch to better choices wherever possible, until nothing improves., TD-GammonTD-Gammon, prop. n.: A backgammon program built at IBM in the early 1990s that taught itself to play at near world-champion level by playing more than a million games against itself., upper confidence boundupper confidence bound, n.: A rule for choosing among uncertain options that gives each one a bonus for how little is known about it, so promising but rarely tried options get a fair chance.