model-based reinforcement learning

Reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. in which the agent uses a model of the environmentenvironment, n.: Everything an AI agent interacts with but does not control — a game, a simulator, a website, the physical world. It answers each of the agent's actions with a new situation and, in reinforcement learning, a score. — a function predicting next statesstate, n.: A snapshot of the situation an AI agent is in — for a chess program, the arrangement of pieces and whose turn it is — containing what it needs to decide what to do next. or observations and rewardsreward, n.: The score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time. from current states and actionsaction, n.: A choice an AI agent makes at one moment — move left, place a stone, type a command — which changes its situation and may earn it a reward. — either given in advance, like the rules of a board game, or learned from experience as a world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real.. The model is used to plan, as in Monte Carlo tree searchMonte Carlo tree search, n.: A way for a game-playing program to choose a move by simulating many possible continuations, focusing on the most promising lines, and picking the move that worked out best on average. or model predictive controlmodel predictive control, n.: A control method that repeatedly plans the best moves a few seconds ahead using a model of the machine, carries out only the first move, then plans again with fresh measurements.; to generate imagined experience for training a policy, as in DynaDyna, prop. n.: An early design in which a learning agent builds a model of its world from real experience, then also practices on imagined experience from that model, squeezing more learning out of each real step. and Dreamer; or to compute gradients through predicted trajectoriestrajectory, n.: The record of what happened as an AI agent acted: the sequence of situations it was in, the actions it took and the rewards it received.. Model-based methods are typically far more sample-efficient than model-freemodel-free reinforcement learning, n.: Learning to act well purely from trial and error, without ever building an internal picture of how the world works — like learning a game by habit rather than by thinking moves ahead. ones and adapt quickly when goals change, but errors in a learned model compound over long horizons and can be exploited by the planner, so practical systems use short rollouts, ensembles that estimate uncertainty, or latent models that predict only what matters for decisions, as MuZeroMuZero, prop. n.: A DeepMind game-playing program from 2019 that mastered Go, chess, shogi and Atari games without being told the rules, by learning its own simplified model of each game to plan with. does.

The model-based agent learned to grasp from one hour of real robot data by planning with its learned dynamics.