Arcade Learning Environment
In plain wordsA software platform that lets AI programs play classic Atari 2600 video games, used since 2012 as a standard test of how well one AI can learn many different tasks from screen images.
An open-source framework, built on the Stella Atari 2600 emulator, that exposes more than a hundred of the console's games as reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. environmentsenvironment, n.: Everything an AI agent interacts with but does not control — a game, a simulator, a website, the physical world. It answers each of the agent's actions with a new situation and, in reinforcement learning, a score.: agents receive 210 × 160-pixel color frames (or the console's 128 bytes of RAM), choose among up to 18 joystick actionsaction, n.: A choice an AI agent makes at one moment — move left, place a stone, type a command — which changes its situation and may earn it a reward. and receive changes in the game score as rewardreward, n.: The score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time.. Its value as a benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time. lies in diversity — one agent, without game-specific engineering, must handle shooters, mazes, sports and puzzles — and in human reference scores for comparison. A standard set of 57 games became the yardstick of deep reinforcement learningdeep reinforcement learning, n.: Reinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images., from the deep Q-networkdeep Q-network, n.: A method, made famous by DeepMind in 2013–2015, in which a neural network learned to play dozens of Atari video games from screen pixels and the score alone, many at human level. to Agent57 (2020), the first agent to exceed the human benchmark on all 57. Protocol details matter: sticky actions, no-op starts and frame skipping curb exploitation of the emulator's determinism, and Atari 100k limits interaction to test sample efficiency. Hard-exploration games such as Montezuma's Revenge drove research on intrinsic motivationintrinsic motivation, n.: Giving an AI agent an inner drive — such as curiosity about surprising things — so it keeps exploring and learning even when the task itself offers no reward yet..
We report human-normalized median scores over the 57 ALE games with sticky actions enabled.
Origin
Created by Marc Bellemare, Yavar Naddaf, Joel Veness and Michael Bowling at the University of Alberta and described in 'The Arcade Learning Environment: An Evaluation Platform for General Agents' (arXiv, 2012; Journal of Artificial Intelligence Research, 2013). Marlos Machado and colleagues revised its evaluation protocol in 2018. It is now maintained by the Farama Foundation and used through GymnasiumGymnasium, prop. n.: A free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym..
Related
- deep Q-networkdeep Q-network, n.: A method, made famous by DeepMind in 2013–2015, in which a neural network learned to play dozens of Atari video games from screen pixels and the score alone, many at human level.
- benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
- GymnasiumGymnasium, prop. n.: A free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym.
- deep reinforcement learningdeep reinforcement learning, n.: Reinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images.
- intrinsic motivationintrinsic motivation, n.: Giving an AI agent an inner drive — such as curiosity about surprising things — so it keeps exploring and learning even when the task itself offers no reward yet.
- MuZeroMuZero, prop. n.: A DeepMind game-playing program from 2019 that mastered Go, chess, shogi and Atari games without being told the rules, by learning its own simplified model of each game to plan with.
Sources
- Bellemare, M. G., Naddaf, Y., Veness, J. & Bowling, M. (2013). The Arcade Learning Environment: An Evaluation Platform for General Agents. Journal of Artificial Intelligence Research 47, 253–279. arXiv:1207.4708. arxiv.org/abs/1207.4708
- Machado, M. C. et al. (2018). Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents. Journal of Artificial Intelligence Research 61, 523–562.
- Badia, A. P. et al. (2020). Agent57: Outperforming the Atari Human Benchmark. arXiv:2003.13350. arxiv.org/abs/2003.13350
Mentioned in
action spaceaction space, n.: The full menu of moves an AI agent is allowed to make — a handful of buttons in a video game, or any combination of motor forces for a robot arm., deep Q-networkdeep Q-network, n.: A method, made famous by DeepMind in 2013–2015, in which a neural network learned to play dozens of Atari video games from screen pixels and the score alone, many at human level., GymnasiumGymnasium, prop. n.: A free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym., sample efficiencysample efficiency, n.: How much practice an AI needs to learn something — an agent that learns a game in a hundred tries is far more sample-efficient than one that needs millions., intrinsic motivationintrinsic motivation, n.: Giving an AI agent an inner drive — such as curiosity about surprising things — so it keeps exploring and learning even when the task itself offers no reward yet., MuZeroMuZero, prop. n.: A DeepMind game-playing program from 2019 that mastered Go, chess, shogi and Atari games without being told the rules, by learning its own simplified model of each game to plan with., Volodymyr MnihVolodymyr Mnih, prop. n.: A DeepMind researcher who led the 2013–2015 work in which one neural network learned to play dozens of Atari video games from screen pixels, launching deep reinforcement learning.