active inference
In plain wordsA theory that living things, and machines built on the same idea, act in order to make the world match their predictions, choosing actions expected to keep them in preferred states and reduce their uncertainty.
A framework derived from the free energy principlefree energy principle, n.: A sweeping theory by the neuroscientist Karl Friston that living things persist by keeping themselves in expected states — in effect, by constantly minimizing surprise about what they sense. in which perception, learning and action are all treated as minimizing free energy under a generative model. Instead of maximizing an external rewardreward, n.: The score an AI system receives after an action, telling it how good the outcome was. Reinforcement learning trains the system to collect as much reward as possible over time. signal as in reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time., an active-inference agent encodes preferences as prior beliefs about the observations it expects to receive and selects policies that minimize expected free energy. This quantity splits into a pragmatic term, favoring preferred outcomes, and an epistemic term, favoring the resolution of uncertainty, so that information-seeking and exploration emerge without separate bonuses. Active inference is used in computational neuroscience and psychiatry to model behavior and in AI and robotics to build agents; whether it offers more than a reformulation of Bayesian decision theory and reinforcement learning is debated.
In the active inference model, the agent looks behind the door because doing so reduces its expected free energy, not because it was rewarded.
Origin
Developed by Karl Friston and colleagues from about 2009, for example in 'Reinforcement Learning or Active Inference?' (PLoS ONE, 2009) and 'Action and Behavior: A Free-Energy Formulation' (Biological Cybernetics, 2010). Thomas Parr, Giovanni Pezzulo and Friston's textbook Active Inference appeared in 2022.
Related
- free energy principlefree energy principle, n.: A sweeping theory by the neuroscientist Karl Friston that living things persist by keeping themselves in expected states — in effect, by constantly minimizing surprise about what they sense.
- predictive processingpredictive processing, n.: The idea that the brain is a prediction machine: it constantly guesses what its senses will report next and learns mainly from its mistakes, so what we perceive is largely its best guess.
- reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
- world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real.
- embodied cognitionembodied cognition, n.: The idea that thinking is not something the brain does on its own: the body, its senses and movements, and its surroundings play a real part in how we think and understand.
- artificial lifeartificial life, n.: The study of life by building lifelike systems — in computer simulations, robots or chemistry — to learn how living things organize themselves, reproduce and evolve, and what other kinds of life could exist.
- reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
Sources
- Friston, K. J., Daunizeau, J. & Kiebel, S. J. (2009). Reinforcement Learning or Active Inference? PLoS ONE 4(7).
- Parr, T., Pezzulo, G. & Friston, K. J. (2022). Active Inference: The Free Energy Principle in Mind, Brain, and Behavior. MIT Press.
Mentioned in
predictive processingpredictive processing, n.: The idea that the brain is a prediction machine: it constantly guesses what its senses will report next and learns mainly from its mistakes, so what we perceive is largely its best guess., free energy principlefree energy principle, n.: A sweeping theory by the neuroscientist Karl Friston that living things persist by keeping themselves in expected states — in effect, by constantly minimizing surprise about what they sense., Karl FristonKarl Friston, prop. n.: A British neuroscientist who proposed that brains, and perhaps all living systems, work by constantly predicting their surroundings and acting to reduce surprise, an idea some see as a blueprint for AI.