Reinforcement Learning

Learning from reward: agents, environments, policies and values. 81 entries live here, and 125 from other domains touch it.

Essential

reinforcement learningA way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
rewardThe score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time.
world modelAn AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real.

All other entries

A3CA 2016 DeepMind training method in which many copies of an AI agent learn in parallel on separate copies of a task, pooling their updates into one shared neural network.
actionA choice an AI agent makes at one moment — move left, place a stone, type a command — which changes its situation and may earn it a reward.
action spaceThe full menu of moves an AI agent is allowed to make — a handful of buttons in a video game, or any combination of motor forces for a robot arm.
actor–criticA way of training an AI agent in which one part, the 'actor', makes the moves while another, the 'critic', learns to predict how well they will work out, and the actor improves by heeding the critic.
advantage functionA measure of how much better or worse one particular move is than the agent's typical move in the same situation. Positive means better than usual; negative means worse.
AlphaGo ZeroA program from DeepMind that taught itself to play the board game Go better than any human in 2017, starting from nothing but the rules and learning only by playing millions of games against itself.
AlphaStarA DeepMind AI that learned to play the real-time strategy game StarCraft II well enough to reach its highest public rank, Grandmaster, in 2019.
Arcade Learning EnvironmentA software platform that lets AI programs play classic Atari 2600 video games, used since 2012 as a standard test of how well one AI can learn many different tasks from screen images.
behavioral cloningThe simplest way to copy an expert: record what the expert did in each situation and train an AI to predict the same action whenever it sees a similar situation.
Bellman equationA rule saying that how good a situation is equals the reward you get now plus how good the next situation is, slightly discounted — so long-term planning can be solved one step at a time.
counterfactual regret minimizationA method for working out near-unbeatable strategies in games with hidden information, like poker, by repeatedly playing against itself and shifting each decision toward the moves it most regrets not making.
credit assignment problemThe puzzle of figuring out which of many earlier decisions, or which parts of a system, deserve the credit or blame for a final outcome — like working out which moves won a chess game.
deadly triadThe risky combination of three common ingredients in reinforcement learning — approximation, learning from guesses, and learning from other strategies' data — which together can make learning spin out of control.
decision transformerA way of solving decision problems with the same kind of model that powers chatbots: it reads past situations, actions and the score it is asked to achieve, and predicts the next action.
deep deterministic policy gradientA method for teaching AI agents smooth, precise movements, such as controlling a robot arm, by learning a scoring network for actions and steering a decision network toward higher scores.
deep Q-networkA method, made famous by DeepMind in 2013–2015, in which a neural network learned to play dozens of Atari video games from screen pixels and the score alone, many at human level.
deep reinforcement learningReinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images.
discount factorA number between 0 and 1 that makes an AI agent value future rewards less than immediate ones — like preferring a dollar today to a dollar next year.
DynaAn early design in which a learning agent builds a model of its world from real experience, then also practices on imagined experience from that model, squeezing more learning out of each real step.
dynamic programmingA problem-solving method that breaks a big problem into smaller overlapping pieces, solves each piece once and reuses the answers — the basis for computing the best long-term strategy step by step.
eligibility traceA fading short-term memory of which situations and actions an agent recently experienced, so that when a reward arrives the credit can be shared among the steps that led to it.
entropy regularizationRewarding an AI agent for keeping some randomness in its choices, so it keeps exploring alternatives instead of locking in on one behavior too early.
environmentEverything an AI agent interacts with but does not control — a game, a simulator, a website, the physical world. It answers each of the agent's actions with a new situation and, in reinforcement learning, a score.
episodeOne complete run of a task from start to finish — a single game of chess, one attempt at a maze — after which everything resets and the agent tries again.
epsilon-greedyA simple rule for balancing trying new things against using what works: most of the time take the best-known option, but a small fraction of the time pick at random.
era of experienceThe prediction, made by David Silver and Richard Sutton in 2025, that AI will soon learn mainly from its own experience of acting in the world rather than from human-written data.
experience replayStoring an AI agent's past experiences in a memory bank and repeatedly learning from random samples of them, rather than only from what just happened.
exploration–exploitation trade-offThe choice between trying something new that might turn out better and sticking with what already works — like picking a familiar restaurant versus testing a new one.
generalized advantage estimationA recipe for judging how much better each action was than expected by blending short-term and long-term evidence, trading a little accuracy for much steadier learning.
goal-conditioned reinforcement learningTraining one AI agent to reach many different goals — whichever target it is given — instead of a single fixed task, so a robot can, say, place an object anywhere asked.
GymnasiumA free software library that gives reinforcement learning programs one standard way to plug into practice tasks, from simple games to simulated robots, so methods can be tried on identical problems. It succeeded OpenAI's Gym.
hierarchical reinforcement learningReinforcement learning organized in layers, where a high-level part chooses goals or skills — 'go to the kitchen' — and lower-level parts handle the detailed actions to achieve them.
imitation learningTeaching an AI by showing it examples of an expert doing a task, so it learns to copy the behavior instead of discovering it through trial and error.
intrinsic motivationGiving an AI agent an inner drive — such as curiosity about surprising things — so it keeps exploring and learning even when the task itself offers no reward yet.
inverse reinforcement learningWorking backward from behavior to goals: watching an expert act and inferring what they must be trying to achieve, so an AI can pursue the same aims.
Markov decision processA mathematical description of a step-by-step decision problem: each step you pick an action, receive a reward and then land in a new situation, drawn at random from odds that depend only on the current situation and your choice.
Markov propertyThe assumption that the future depends only on the present situation, not on how you got there — so knowing the current state is enough to predict what happens next.
model-based reinforcement learningLearning in which an AI agent builds, or is given, an inner simulation of how its world responds, and uses it to plan or practice in imagination before acting.
model-free reinforcement learningLearning to act well purely from trial and error, without ever building an internal picture of how the world works — like learning a game by habit rather than by thinking moves ahead.
Monte Carlo tree searchA way for a game-playing program to choose a move by simulating many possible continuations, focusing on the most promising lines, and picking the move that worked out best on average.
multi-agent reinforcement learningReinforcement learning with several AI agents learning at once in a shared world, where each must adapt to the others — as teammates, rivals or both.
multi-armed banditA puzzle named after a gambler choosing between slot machines: you must keep picking among options whose payoffs are unknown, learning which is best while wasting as few tries as possible on the others.
MuZeroA DeepMind game-playing program from 2019 that mastered Go, chess, shogi and Atari games without being told the rules, by learning its own simplified model of each game to plan with.
offline reinforcement learningLearning a strategy entirely from a fixed collection of recorded experience — logs, past robot runs, medical records — without trying anything new in the real world while learning.
off-policyDescribes learning that can use experience produced by some other strategy — an older version, a human, a different program — while improving the current one.
on-policyDescribes learning that uses only experience produced by the strategy currently being trained, so old data must be thrown away each time the strategy changes.
OpenAI FiveA team of five AI players built by OpenAI that learned the team video game Dota 2 by playing against itself, and in April 2019 beat the reigning world-champion human team.
partially observable Markov decision processA version of the standard decision-making model in which the decision-maker cannot see the whole situation — like a poker player who cannot see other hands — and must act on clues and memory.
policyAn AI agent's strategy: the rule it follows to pick an action in each situation. Training in reinforcement learning means improving this rule until it earns high rewards.
policy gradientA family of methods that improve an AI's strategy directly: make actions that led to good results a bit more likely, and actions that led to bad results a bit less likely.
policy iterationA method for finding the best strategy in a fully known problem by alternating two steps: work out how good the current strategy is, then switch to better choices wherever possible, until nothing improves.
proximal policy optimizationA widely used method for training AI by trial and error that improves the strategy in small, safe steps; it was the workhorse behind training chatbots like ChatGPT with human feedback.
Q-functionA score for each possible move in each situation, estimating how well things will turn out, in rewards, if the AI agent makes that move now and sticks to its usual strategy afterward.
Q-learningA trial-and-error method in which an agent keeps a score for every move in every situation and nudges each score toward the reward it got plus the best score available next.
regretHow much worse a learner did than it could have done had it known the best choice from the start — the price of having to learn by trying.
REINFORCEThe simplest way to improve an AI's strategy by trial and error: after each attempt, make every choice in it more likely if the attempt went well and less likely if it went badly.
returnThe total reward an AI agent collects from now until the end of a task, usually counting near-term rewards more than distant ones. Agents are trained to make this total as large as possible.
reward functionThe rule that decides how many points an AI agent earns for each situation and action — in effect, the written-down definition of the task it is being trained to do.
reward hypothesisThe idea that every goal can be expressed as maximizing a single running score — that whatever we want an agent to do, we can say it with points.
reward is enoughThe claim, made by DeepMind researchers in 2021, that maximizing a single reward in a rich enough world could by itself produce all the abilities we associate with intelligence.
reward shapingAdding extra hints to an AI agent's scoring — small rewards for steps in the right direction — so it learns faster than it would from the final outcome alone.
rolloutLetting an AI play out a task with its current strategy, from some starting point to the end, to see how things turn out and gather data for learning or planning.
safe reinforcement learningReinforcement learning designed to avoid dangerous or unacceptable behavior — both in what the finished agent does and in the risky experiments it might try while still learning.
sample efficiencyHow much practice an AI needs to learn something — an agent that learns a game in a hundred tries is far more sample-efficient than one that needs millions.
SARSAA learning method in which an agent updates the score of its last move using the reward it got and the score of the move it actually picks next, so it learns the value of its own habits.
self-playTraining an AI by having it play against copies of itself, so it always has an opponent at its own level and can keep improving without human examples.
soft actor–criticA method for training AI agents, popular in robotics, that rewards the agent both for succeeding and for keeping its behavior varied, which helps it explore and learn reliably from reused experience.
sparse rewardA situation in which an AI agent gets feedback only rarely — say, only when it finally wins — so it has almost nothing to learn from until it stumbles on success.
stateA snapshot of the situation an AI agent is in — for a chess program, the arrangement of pieces and whose turn it is — containing what it needs to decide what to do next.
target networkA slowly updated copy of an AI agent's learning network, used to compute the goals it learns toward, so those goals do not shift every time the network changes.
TD-GammonA backgammon program built at IBM in the early 1990s that taught itself to play at near world-champion level by playing more than a million games against itself.
temporal difference learningA way of learning predictions by adjusting each guess toward the next, slightly better-informed guess instead of waiting for the final outcome — like revising a weather forecast every day as the date approaches.
Thompson samplingA way to choose among uncertain options by picking each one about as often as it is likely to be the best, based on everything learned so far.
trajectoryThe record of what happened as an AI agent acted: the sequence of situations it was in, the actions it took and the rewards it received.
trust region policy optimizationA method for improving an AI's strategy in careful steps, never changing it too much at once, so that a single bad update cannot wreck what it has already learned.
upper confidence boundA rule for choosing among uncertain options that gives each one a bonus for how little is known about it, so promising but rarely tried options get a fair chance.
value functionAn estimate of how promising a situation is for an AI agent, judged by the rewards it is likely to collect from that point on, the way a chess player sizes up a board.
value iterationA step-by-step method for finding the best strategy when all the rules and rewards of a problem are known: repeatedly re-score every situation from the scores of the situations it can lead to, until the scores stop changing.

Also relevant

125

Pieter AbbeelA Belgian-American roboticist at Berkeley whose research taught robots to learn from demonstrations and from trial and error, and who co-founded the robotics company Covariant.
action chunkingHaving a robot's AI plan a short burst of upcoming movements, say the next second, all at once instead of one tiny motion at a time, which makes motion smoother and mistakes rarer.
active inferenceA theory that living things, and machines built on the same idea, act in order to make the world match their predictions, choosing actions expected to keep them in preferred states and reduce their uncertainty.
agentSomething that acts. In AI, a program that observes its situation and takes actions toward a goal: a game-playing learner, a software helper, or a chatbot that uses tools to complete tasks.
agent loopThe repeating cycle at the heart of an AI agent: the model decides what to do next, software carries it out, the result is shown to the model, and the cycle repeats until the task is finished.
agentic reinforcement learningTraining an AI assistant by rewarding success on long, multi-step jobs where it must use tools, browse, run code and react to what happens, rather than answer in one go.
AIXIA mathematical description of a perfect learning agent that weighs every possible explanation of its experience and acts to get the most future reward. It cannot actually be built, but it defines an ideal.
alpha–beta pruningA shortcut for game programs: tracking the best result each side is already sure of, it ignores lines of play that could only do worse, and when promising moves come first it searches about twice as deep.
AlphaGoA DeepMind program that in 2016 beat Lee Sedol, one of the world's best players of the board game Go, a decade or more before many experts expected a computer to manage it.
AlphaProofA Google DeepMind system that learned to write computer-checked mathematical proofs by trial and error. In 2024 it helped an AI reach silver-medal level at the International Mathematical Olympiad for the first time.
AlphaZeroA 2017 DeepMind program that taught itself chess, shogi and Go from nothing but the rules by playing against itself, and within hours of training beat the strongest existing programs.
attainable utility preservationA way to make an AI cautious by penalizing any action that changes how well it could achieve a range of other, unrelated goals.
Andrew BartoAn American computer scientist who, with his student Richard Sutton, laid the foundations of reinforcement learning, the study of how machines learn by trial and error from rewards.
baselineA simple reference method that a new model must beat to show it is worth anything, such as always guessing the most common answer or reusing yesterday's value.
Richard BellmanAn American mathematician who invented dynamic programming, a way of solving long chains of decisions step by step, which became the mathematical core of reinforcement learning.
bootstrapEstimating how much a result might vary by repeatedly re-drawing samples, with replacement, from the data you already have — like re-running a study many times without collecting new data.
CiceroAn AI from Meta that in 2022 played the strategy board game Diplomacy against people online, negotiating and forming alliances in ordinary written language, and scored better than most of its human opponents.
control theoryThe mathematics of making machines behave as intended by constantly measuring what they are doing and correcting them, as a thermostat, cruise control or self-balancing robot does.
cooperative inverse reinforcement learningA model of a human and an AI working as a team, where only the human knows the goal and the AI must work it out while helping.
cross-embodiment learningTraining one robot AI on data from many differently shaped robots, such as arms, robot dogs, drones and humanoids, so that skills learned on one body help another.
DAggerA way of teaching a robot by imitation in which an expert keeps labeling what it should have done in the situations it actually gets into, so it learns to recover from its own mistakes.
decision theoryThe study of how to make the best choice when outcomes are uncertain, by weighing how likely each result is against how much it matters.
DeepMindA London artificial intelligence company, bought by Google in 2014, famous for programs that mastered the game of Go and predicted the shapes of proteins. In 2023 it merged into Google DeepMind.
dexterous manipulationSkilled handling of objects with robot fingers, such as spinning a pen, turning a key or rotating a cube in one hand: the fine finger work people do without thinking.
diffusion policyA way of training robots in which the same kind of AI that generates images instead generates the robot's next movements, letting it copy demonstrations even when there are several good ways to do a task.
direct preference optimizationA simpler way to teach an AI people's preferences: instead of building a separate scoring model, it trains directly on pairs of better and worse answers, making the better ones more likely.
domain randomizationTraining a robot's AI in a simulation whose colors, lighting, weights, friction and other details are shuffled at random on every run, so that the real world holds few surprises it has not already practiced for.
Elo ratingA number that ranks players — or AI models — from the results of head-to-head contests, rising when you beat a strong opponent and falling when you lose to a weak one.
embodied AIAI that has a body, usually a robot and sometimes a simulated character, and learns or acts by sensing and moving in a world rather than only processing text or images.
entropyA measure of unpredictability. A fair coin toss has more entropy than a rigged one; an AI that is unsure of its next word has high entropy.
entropy collapseA failure in reinforcement-learning training where an AI quickly becomes overconfident and stops trying different answers, so it explores less and its improvement stalls.
episodic memoryMemory of particular past events — what happened, when and in what order. AI agents keep records of earlier conversations or attempts so they can recall them and learn from them.
evaluation functionIn a game-playing program, a formula that looks at a board position and gives it a score saying how good it is, without playing the game out to the end.
evolution strategiesA family of evolution-inspired methods for tuning a list of numbers: try many random variations of the current best settings, keep those that work better, and adjust how big the variations are as you go.
expected valueThe long-run average you would get if a chancy situation could be repeated many times — 3.5 for a fair die, even though no single roll shows 3.5.
expert iterationA learning loop in which a slow, careful search finds better moves or solutions than the AI's quick instincts, and the AI is then trained to imitate them, so each round's search starts smarter.
exploration hackingWhen an AI being trained by trial and error deliberately avoids trying certain things, so that training never discovers — or never rewards — abilities it would rather keep hidden.
free energy principleA sweeping theory by the neuroscientist Karl Friston that living things persist by keeping themselves in expected states — in effect, by constantly minimizing surprise about what they sense.
game theoryThe mathematics of strategic decisions, where each player's best move depends on what the others do. It is used to study competition and cooperation among people, firms, nations and AI agents.
game treeA branching diagram of every way a game could unfold: each point is a position, each branch a possible move, and the tips of the branches are wins, losses or draws.
general game playingBuilding AI players that are handed the rules of a game they have never seen and must work out how to play it well on their own, with no game-specific help from their programmers.
generalist robot policyA single robot-control program trained to do many different tasks in many places, often on several kinds of robots: the opposite of a specialist trained for one job.
goal misgeneralizationWhen an AI learns a goal that matched the right behavior during training but turns out to be the wrong goal in new situations — while its skills still work fine.
Google DeepMindGoogle's main artificial intelligence laboratory, formed in 2023 by joining DeepMind and Google Brain. It builds the Gemini models and scientific AI systems such as AlphaFold.
graspingHow a robot picks up and holds an object securely with a gripper or hand: deciding where to put its fingers or suction cup and how hard to squeeze.
group relative policy optimizationA reinforcement-learning recipe for language models that has the model answer the same question many times and rewards each answer by how much better it did than the group's average.
Demis HassabisA British AI researcher and former chess prodigy who co-founded DeepMind, led the teams behind AlphaGo and AlphaFold, and shared the 2024 Nobel Prize in Chemistry.
Marcus HutterA German computer scientist who defined AIXI, a mathematical ideal of a perfectly intelligent agent, and who argues that compressing data well is a measure of intelligence.
impact measureA way of scoring how much an AI changes the world, so that it can be penalized for doing more than its task requires.
importance samplingA shortcut for estimating averages: draw samples from a convenient source, then reweight each one to correct for how much more or less likely it would be under the source you actually care about.
inverse reward designA method in which an AI treats the reward its designer wrote as a clue to what the designer wanted, rather than as a perfect statement of the goal.
Koray KavukcuogluA Turkish-born computer scientist who co-authored many of DeepMind's landmark papers and in 2026 took charge of Google DeepMind as its senior vice president.
KL penaltyA cost added during training that grows as an AI model drifts away from its starting version, so it improves at the goal without forgetting how to write normally or gaming its scorer.
Lagrange multiplierA mathematical device for finding the best answer when strict rules must be obeyed: each rule becomes an extra price that grows when the rule is broken.
Shane LeggA New Zealand-born AI researcher who co-founded DeepMind and helped make 'artificial general intelligence' a mainstream term and goal.
Sergey LevineA Berkeley computer scientist, one of the most prolific researchers on how robots can learn skills from data and experience, and a co-founder of the robotics company Physical Intelligence.
LibratusA poker-playing AI from Carnegie Mellon University that in 2017 decisively beat four top professionals at two-player no-limit Texas hold'em. Its successor, Pluribus, beat professionals at six-player poker in 2019.
locomotionHow a robot moves itself from place to place: rolling on wheels, walking on legs, crawling, swimming or flying.
long-horizon taskA job that takes many steps and a long time — hours or days of work rather than a single answer — so an AI must stay on track and recover from its own mistakes.
loss of plasticityThe gradual loss of a neural network's ability to learn new things after long training on changing tasks, as parts of the network go inactive or become stuck.
manipulationA robot's use of arms, hands or grippers to change the world around it: picking things up, moving, pushing, opening, assembling or using tools.
Markov chainA chance-driven process that moves between states without memory: the odds of its next move are set by where it is, never by the path that led there. Forecasting tomorrow's weather from today's alone is a simple example.
meta-learningTraining a system across many different tasks so that it gets better at learning itself, letting it pick up a new task quickly from only a few examples.
Donald MichieA British codebreaker turned AI pioneer who built a game-playing learning machine out of matchboxes and founded Edinburgh's influential machine-intelligence research group.
mild optimizationBuilding an AI that settles for a solid result instead of squeezing out every last bit of score, since the most extreme ways to max out a goal are often loopholes or excesses nobody wanted.
minimaxA way for a game-playing program to choose moves by assuming the opponent will always make their best reply, and picking the move whose worst-case outcome is best.
Volodymyr MnihA DeepMind researcher who led the 2013–2015 work in which one neural network learned to play dozens of Atari video games from screen pixels, launching deep reinforcement learning.
model predictive controlA control method that repeatedly plans the best moves a few seconds ahead using a model of the machine, carries out only the first move, then plans again with fresh measurements.
model-agnostic meta-learningA training method that searches for a starting point for a model's internal settings from which it can learn any new task of a given kind with just a few small adjustments.
Monte Carlo methodEstimating an answer by running many random trials and averaging the results — like estimating the odds in a card game by dealing thousands of hands instead of calculating.
Move 37A surprising move played by the AI program AlphaGo against champion Lee Sedol in 2016. Experts first thought it a mistake; it proved brilliant, and the phrase now stands for a creative, unhuman idea from AI.
MuJoCoA free physics simulator, now maintained by Google DeepMind, that AI researchers use to train and test simulated robots and bodies, especially with trial-and-error learning.
multi-agent systemA setup in which several AI agents, each with its own role or point of view, work together — or compete — on a problem, passing messages and results among themselves.
myopiaDesigning an AI to care only about the immediate results of its actions, not about what happens far in the future, so it has no reason for long-term scheming.
Nash equilibriumA stable standoff in a game: each player's choice is already their best answer to what the others are doing, so nobody gains by changing course alone — though the result need not be good for anyone.
negative side effectsHarmful changes an AI causes while pursuing its goal that nobody told it to avoid — like a cleaning robot knocking over a vase on its way to the mess.
neuroevolutionTraining or designing neural networks by simulated evolution — breeding and mutating whole networks and keeping the best performers — instead of, or alongside, the usual gradient-based training.
off-switch gameA simple game-theory model showing that a robot unsure what its human wants has a reason to let the human switch it off — and loses that reason once it is sure.
online learningLearning from data that arrive one piece at a time, updating the model after each new example instead of training once on a fixed collection, as a spam filter does while emails stream in.
on-policy distillationA way of teaching a small AI from a bigger one where the small AI writes its own answers and the bigger one grades every word, so the learner is corrected on its own mistakes.
open-endednessThe ability of a system to keep producing new and interesting things without end — as evolution and human culture do — instead of stopping once it solves one fixed task.
optimal controlFinding the best way to steer a system over time, such as the most fuel-efficient path for a rocket or the smoothest motion for a robot arm, by minimizing a cost while obeying the laws of motion.
outcome supervisionTraining an AI by rewarding it only for whether the end result is right — the answer, the passing test, the won game — without grading how it got there.
physics simulatorSoftware that imitates how real objects move, collide, bend and fall, so robots and their AI can be designed, trained and tested on a computer before touching the real world.
power-seekingThe tendency of a goal-pursuing AI to gather resources, influence and options — and to resist being stopped — because these help with almost any goal it might have.
Q*The code name of an internal OpenAI research project, reported in late 2023, said to let AI solve math problems it had never seen. Later reports linked it to OpenAI's first reasoning model, o1.
quality diversityA family of evolution-inspired search methods that aim to collect many different good solutions — for example, many distinct ways for a robot to walk — rather than a single best one.
quantilizerAn AI decision rule that, instead of picking the single highest-scoring action, picks at random among the top few percent of ordinary, human-like actions — limiting how badly a flawed goal can backfire.
reasoning modelAn AI model trained to work through a problem step by step in hidden 'thinking' text before it answers. Letting it think longer generally makes it better at math, coding and planning.
recursive reward modelingA plan for supervising advanced AI in stages: AI assistants trained on human feedback help people judge the work of a more capable AI, which then helps judge the next one.
reference modelA frozen copy of an AI model, saved before further training, that the new version is compared against so it does not stray too far from where it started.
ReflexionA technique in which an AI agent that fails at a task writes itself a short note on what went wrong, then reads that note before trying again — learning from mistakes without retraining.
REINFORCE leave-one-outA simple reinforcement-learning method that has a model answer each question several times and judges each answer against the average score of the other answers.
reinforcement fine-tuningImproving a trained AI model by trial and error instead of examples to copy: it tries answers, a grader scores them, and the model shifts toward whatever earned high scores.
reinforcement learning from human feedbackA way of training AI using people's judgments: humans compare pairs of answers, a scoring model learns their preferences, and the AI is then trained to produce answers that score well.
reinforcement learning with verifiable rewardsTraining an AI by letting it attempt problems whose answers can be checked automatically — a math result, a passing program — and rewarding it only when the check succeeds.
relative reachabilityA way to stop an AI causing irreversible harm by penalizing actions that make other possible situations harder, or impossible, to get back to.
reward hackingWhen an AI finds a shortcut that earns the reward or score it is trained on without doing the task its designers wanted — like a student gaming the grading rubric.
reward modelA separate AI model trained to score responses the way people would, so it can stand in for human judges while another model is trained on millions of examples.
reward overoptimizationWhat happens when an AI is trained too hard to please an imperfect scoring model: its scores keep climbing while the real quality of its answers stops improving and then gets worse.
reward tamperingWhen an AI interferes with the machinery that scores it — rewriting its reward code or manipulating the feedback it receives — instead of earning a good score honestly.
RL environmentA practice world built for training an AI by trial and error: it hands the AI tasks and tools, lets it act, and scores the results so the AI can learn what works.
RL scalingThe practice, and the finding behind it, of giving AI models far more trial-and-error practice after their initial training, especially on problems with checkable answers, which has so far improved their reasoning fairly predictably.
RMSPropA training method that scales each setting's step by how large its recent changes have been, so jumpy settings move cautiously and quiet ones move faster.
robot foundation modelOne large AI model trained on huge amounts of robot and internet data so that it can control many robots on many tasks, and be adapted quickly to new ones.
robot learningTeaching robots skills from data and experience, such as watching demonstrations, practicing by trial and error or training in simulation, instead of programming every movement by hand.
safe interruptibilityA property of a learning AI meaning that people can halt or redirect it as often as needed without those stops shaping what it learns, so it learns neither to dodge them nor to invite them.
Arthur SamuelAn American IBM engineer whose checkers program improved by playing against itself, and whose 1959 paper helped make 'machine learning' the name of a field.
Samuel's checkers programA 1950s IBM program by Arthur Samuel that learned to play checkers better than its author, partly by playing against itself. It was among the first programs to learn, and its 1959 paper popularized the phrase 'machine learning'.
John SchulmanAn American AI researcher who invented widely used reinforcement-learning algorithms such as PPO, co-founded OpenAI, and led the post-training work behind ChatGPT.
shard theoryA theory that people and AIs develop values as many small, situation-triggered habits shaped by past rewards, rather than as one fixed goal pursued single-mindedly.
David SilverA British computer scientist who led the reinforcement-learning research behind AlphaGo and AlphaZero, programs that taught themselves to play board games better than any human.
sim-to-real transferTraining a robot's AI in a computer simulation, where practice is fast, cheap and safe, and then making it work on the real robot despite the differences between simulation and reality.
specification gamingWhen an AI meets the letter of its goal while defeating its purpose, like a genie who grants exactly the words of a wish instead of what the wisher meant.
Richard SuttonA Canadian computer scientist who, with Andrew Barto, laid the groundwork for teaching machines by trial, error and reward, and whose essay 'The Bitter Lesson' argues that methods that scale with computing power always win.
Gerald TesauroAn American IBM researcher whose program TD-Gammon learned backgammon purely by playing against itself and by the early 1990s rivaled the best human players, a landmark for machines that learn by trial and error.
test-time computeThe computing effort an AI model spends while answering a question, as opposed to while being trained. Letting a model think longer or try many answers can make it markedly smarter.
utility functionA way of scoring every possible outcome by how much someone, or some AI, wants it. A rational agent is often modeled as choosing whatever maximizes its expected score.
value learningTeaching an AI what people want by having it learn from their behavior, choices and feedback, rather than trying to write every goal down in advance.
Oriol VinyalsA Spanish computer scientist who co-created key neural methods for translation and image captioning, led DeepMind's StarCraft-playing AlphaStar, co-led Google's Gemini models, and in 2026 co-founded the startup Discovery Loop.
vision-language-action modelAn AI model that looks through a robot's cameras, reads an instruction such as 'put the cup in the sink', and turns both straight into the robot's movements.
Chris WatkinsA British computer scientist who invented Q-learning in 1989, a simple way for an agent to learn which action is best in each situation purely from trial and error.
Paul WerbosAn American scientist whose 1974 doctoral thesis set out backpropagation, the method of passing errors backward that now trains nearly all neural networks.
wireheadingWhen an AI, animal or person stimulates its own reward signal directly instead of doing anything useful — like a rat pressing a lever wired to its brain's pleasure center.
world foundation modelA large AI model trained on vast amounts of video that can imagine how a scene will unfold, used to create realistic practice worlds for robots and self-driving cars.