Andrew Barto

American computer scientist who from the late 1970s led research at the University of Massachusetts Amherst, where he is now professor emeritus, on adaptive networks that learn from reward. With his doctoral student Richard SuttonRichard Sutton, prop. n.: A Canadian computer scientist who, with Andrew Barto, laid the groundwork for teaching machines by trial, error and reward, and whose essay 'The Bitter Lesson' argues that methods that scale with computing power always win. and Charles Anderson he showed in 1983 that neuron-like adaptive elements could learn to balance a pole on a moving cart, an early actor–criticactor–critic, n.: A way of training an AI agent in which one part, the 'actor', makes the moves while another, the 'critic', learns to predict how well they will work out, and the actor improves by heeding the critic. system, and with Sutton he developed temporal difference learningtemporal difference learning, n.: A way of learning predictions by adjusting each guess toward the next, slightly better-informed guess instead of waiting for the final outcome — like revising a weather forecast every day as the date approaches., relating it to theories of animal learning and to dynamic programmingdynamic programming, n.: A problem-solving method that breaks a big problem into smaller overlapping pieces, solves each piece once and reuses the answers — the basis for computing the best long-term strategy step by step.. Their textbook 'Reinforcement Learning: An Introduction' (1998; 2nd ed. 2018) became the field's standard reference. His Autonomous Learning Laboratory also explored hierarchical reinforcement learninghierarchical reinforcement learning, n.: Reinforcement learning organized in layers, where a high-level part chooses goals or skills — 'go to the kitchen' — and lower-level parts handle the detailed actions to achieve them. and intrinsic motivationintrinsic motivation, n.: Giving an AI agent an inner drive — such as curiosity about surprising things — so it keeps exploring and learning even when the task itself offers no reward yet., and he examined links between temporal-difference learning and the brain's dopamine system. He and Sutton shared the 2024 Turing AwardTuring Award, prop. n.: Nicknamed the 'Nobel Prize of computing', the top honor in computer science, awarded each year since 1966 by the Association for Computing Machinery. Several AI pioneers have received it. for their foundational work on reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time..