Richard Sutton

American-born Canadian computer scientist who, with his doctoral adviser and longtime collaborator Andrew BartoAndrew Barto, prop. n.: An American computer scientist who, with his student Richard Sutton, laid the foundations of reinforcement learning, the study of how machines learn by trial and error from rewards., built much of the theory and many of the core algorithms of modern reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.. In his doctoral work with Barto at the University of Massachusetts Amherst and in a 1988 paper he developed temporal difference learningtemporal difference learning, n.: A way of learning predictions by adjusting each guess toward the next, slightly better-informed guess instead of waiting for the final outcome — like revising a weather forecast every day as the date approaches., in which predictions are updated toward later predictions, and with Barto he introduced actor–criticactor–critic, n.: A way of training an AI agent in which one part, the 'actor', makes the moves while another, the 'critic', learns to predict how well they will work out, and the actor improves by heeding the critic. methods (1983). Later contributions include the Dyna architecture combining learning and planning (1990), the options framework for hierarchical reinforcement learninghierarchical reinforcement learning, n.: Reinforcement learning organized in layers, where a high-level part chooses goals or skills — 'go to the kitchen' — and lower-level parts handle the detailed actions to achieve them. (1999) and the policy gradientpolicy gradient, n.: A family of methods that improve an AI's strategy directly: make actions that led to good results a bit more likely, and actions that led to bad results a bit less likely. theorem (with David McAllester, Satinder Singh and Yishay Mansour, 2000). Their textbook 'Reinforcement Learning: An Introduction' (1998; 2nd ed. 2018) defined the field. A professor at the University of Alberta since 2003, he has also worked at DeepMind and at Keen Technologies. His essay 'The Bitter Lessonthe bitter lesson, phr.: The argument, made by AI pioneer Rich Sutton in 2019, that AI has advanced mostly through general methods that use ever more computing power, not by programming in human knowledge.' (2019) argued that general methods that exploit growing computation beat built-in human knowledge. He shared the 2024 Turing AwardTuring Award, prop. n.: Nicknamed the 'Nobel Prize of computing', the top honor in computer science, awarded each year since 1966 by the Association for Computing Machinery. Several AI pioneers have received it. with Barto.