self-play
In plain wordsTraining an AI by having it play against copies of itself, so it always has an opponent at its own level and can keep improving without human examples.
A training method for agents in competitive or cooperative multi-agent settings in which the learner generates its own experience by interacting with copies of itself — its current version or a pool of past versions — rather than with fixed opponents or human data. Because the opponent improves as the agent does, self-play provides an automatic curriculum of steadily harder challenges and can surpass human performance, as in TD-GammonTD-Gammon, prop. n.: A backgammon program built at IBM in the early 1990s that taught itself to play at near world-champion level by playing more than a million games against itself., AlphaGo Zero and AlphaZeroAlphaZero, prop. n.: A 2017 DeepMind program that taught itself chess, shogi and Go from nothing but the rules by playing against itself, and within hours of training beat the strongest existing programs., which learned Go, chess and shogi from random play with Monte Carlo tree searchMonte Carlo tree search, n.: A way for a game-playing program to choose a move by simulating many possible continuations, focusing on the most promising lines, and picking the move that worked out best on average.. Naive self-play can cycle, as in rock–paper–scissors, or overfit to its own quirks, so systems use opponent pools, fictitious self-play (best responses to the historical average), population-based leagues with dedicated exploiter agents (AlphaStar) or game-theoretic methods such as counterfactual regret minimizationcounterfactual regret minimization, n.: A method for working out near-unbeatable strategies in games with hidden information, like poker, by repeatedly playing against itself and shifting each decision toward the moves it most regrets not making.. The term also covers large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. training in which a model sets problems for itself, or debates or negotiates with copies of itself.
After three days of self-play the new agent beat the version trained on human games 100 to 0.
Origin
Arthur Samuel's checkers program (1959) improved partly by playing against a copy of itself, and Gerald Tesauro's TD-Gammon (1992) learned backgammon almost entirely through self-play. The approach became famous with DeepMind's AlphaGo Zero (2017), which learned without human games; fictitious self-play (Johannes Heinrich, Marc Lanctot and David Silver, 2015) extended it to imperfect-information games.
Usage note
Self-play works best where outcomes are objectively scored, as with a game result or a verified proof; without such a signal, a model judging its own output can reinforce its own errors.
Related
- AlphaZeroAlphaZero, prop. n.: A 2017 DeepMind program that taught itself chess, shogi and Go from nothing but the rules by playing against itself, and within hours of training beat the strongest existing programs.
- TD-GammonTD-Gammon, prop. n.: A backgammon program built at IBM in the early 1990s that taught itself to play at near world-champion level by playing more than a million games against itself.
- Monte Carlo tree searchMonte Carlo tree search, n.: A way for a game-playing program to choose a move by simulating many possible continuations, focusing on the most promising lines, and picking the move that worked out best on average.
- multi-agent reinforcement learningmulti-agent reinforcement learning, n.: Reinforcement learning with several AI agents learning at once in a shared world, where each must adapt to the others — as teammates, rivals or both.
- counterfactual regret minimizationcounterfactual regret minimization, n.: A method for working out near-unbeatable strategies in games with hidden information, like poker, by repeatedly playing against itself and shifting each decision toward the moves it most regrets not making.
- self-improvementself-improvement, n.: When an AI gets better by learning from material it produces itself — its own answers, critiques or practice problems — rather than only from new human-made examples.
- curriculum learningcurriculum learning, n.: Teaching a model the way schools teach people: starting with easy examples and gradually moving to harder ones, instead of presenting everything in random order.
Sources
- Silver, D. et al. (2017). Mastering the game of Go without human knowledge. Nature 550, 354–359.
- Samuel, A. L. (1959). Some Studies in Machine Learning Using the Game of Checkers. IBM Journal of Research and Development 3(3), 210–229.
- Heinrich, J., Lanctot, M. & Silver, D. (2015). Fictitious Self-Play in Extensive-Form Games. ICML 2015.
Mentioned in
AlphaGoAlphaGo, prop. n.: A DeepMind program that in 2016 beat Lee Sedol, one of the world's best players of the board game Go, a decade or more before many experts expected a computer to manage it., AlphaGo ZeroAlphaGo Zero, prop. n.: A program from DeepMind that taught itself to play the board game Go better than any human in 2017, starting from nothing but the rules and learning only by playing millions of games against itself., AlphaZeroAlphaZero, prop. n.: A 2017 DeepMind program that taught itself chess, shogi and Go from nothing but the rules by playing against itself, and within hours of training beat the strongest existing programs., Arthur SamuelArthur Samuel, prop. n.: An American IBM engineer whose checkers program improved by playing against itself, and whose 1959 paper helped make 'machine learning' the name of a field., David SilverDavid Silver, prop. n.: A British computer scientist who led the reinforcement-learning research behind AlphaGo and AlphaZero, programs that taught themselves to play board games better than any human., Elo ratingElo rating, n.: A number that ranks players — or AI models — from the results of head-to-head contests, rising when you beat a strong opponent and falling when you lose to a weak one., game theorygame theory, n.: The mathematics of strategic decisions, where each player's best move depends on what the others do. It is used to study competition and cooperation among people, firms, nations and AI agents., minimaxminimax, n.: A way for a game-playing program to choose moves by assuming the opponent will always make their best reply, and picking the move whose worst-case outcome is best., Monte Carlo tree searchMonte Carlo tree search, n.: A way for a game-playing program to choose a move by simulating many possible continuations, focusing on the most promising lines, and picking the move that worked out best on average., Move 37Move 37, prop. n.: A surprising move played by the AI program AlphaGo against champion Lee Sedol in 2016. Experts first thought it a mistake; it proved brilliant, and the phrase now stands for a creative, unhuman idea from AI., multi-agent reinforcement learningmulti-agent reinforcement learning, n.: Reinforcement learning with several AI agents learning at once in a shared world, where each must adapt to the others — as teammates, rivals or both., self-improvementself-improvement, n.: When an AI gets better by learning from material it produces itself — its own answers, critiques or practice problems — rather than only from new human-made examples., AI debateAI debate, n.: A proposed way to supervise very capable AI: two AI systems argue opposite answers to a question and a human judge decides who argued better, on the theory that lies are easier to expose than to defend., AlphaStarAlphaStar, prop. n.: A DeepMind AI that learned to play the real-time strategy game StarCraft II well enough to reach its highest public rank, Grandmaster, in 2019., counterfactual regret minimizationcounterfactual regret minimization, n.: A method for working out near-unbeatable strategies in games with hidden information, like poker, by repeatedly playing against itself and shifting each decision toward the moves it most regrets not making., era of experienceera of experience, n.: The prediction, made by David Silver and Richard Sutton in 2025, that AI will soon learn mainly from its own experience of acting in the world rather than from human-written data., expert iterationexpert iteration, n.: A learning loop in which a slow, careful search finds better moves or solutions than the AI's quick instincts, and the AI is then trained to imitate them, so each round's search starts smarter., general game playinggeneral game playing, n.: Building AI players that are handed the rules of a game they have never seen and must work out how to play it well on their own, with no game-specific help from their programmers., Gerald TesauroGerald Tesauro, prop. n.: An American IBM researcher whose program TD-Gammon learned backgammon purely by playing against itself and by the early 1990s rivaled the best human players, a landmark for machines that learn by trial and error., MuZeroMuZero, prop. n.: A DeepMind game-playing program from 2019 that mastered Go, chess, shogi and Atari games without being told the rules, by learning its own simplified model of each game to plan with., Nash equilibriumNash equilibrium, n.: A stable standoff in a game: each player's choice is already their best answer to what the others are doing, so nobody gains by changing course alone — though the result need not be good for anyone., OpenAI FiveOpenAI Five, prop. n.: A team of five AI players built by OpenAI that learned the team video game Dota 2 by playing against itself, and in April 2019 beat the reigning world-champion human team., open-endednessopen-endedness, n.: The ability of a system to keep producing new and interesting things without end — as evolution and human culture do — instead of stopping once it solves one fixed task., Oriol VinyalsOriol Vinyals, prop. n.: A Spanish computer scientist who co-created key neural methods for translation and image captioning, led DeepMind's StarCraft-playing AlphaStar, co-led Google's Gemini models, and in 2026 co-founded the startup Discovery Loop.