loss of plasticity
In plain wordsThe gradual loss of a neural network's ability to learn new things after long training on changing tasks, as parts of the network go inactive or become stuck.
The decline, over prolonged training on a sequence of tasks or a shifting data stream, of a network's capacity to learn from new data; in reported experiments, performance on each new task eventually sinks to that of a much simpler network, or learning stalls altogether. It is the other half of the continual-learning problem from catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on.: forgetting loses the old, while loss of plasticity blocks the new. Contributing mechanisms include growing numbers of dead or saturated units (for example dead ReLUsdying ReLU, n.: A training failure in which some of a network's units get stuck permanently switched off, always outputting zero, so they stop learning and become dead weight.), growing weight magnitudes, collapsing effective rank of representations, and sharpening of the loss landscapeloss landscape, n.: A way of picturing training as a journey across hilly terrain, where position stands for the model's settings and height for its error. Training searches for low valleys.. It affects deep reinforcement learningdeep reinforcement learning, n.: Reinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images., where the data distribution shifts as the policy improves, and warm-started training in general. Remedies include periodically reinitializing low-utility units (continual backpropagation), shrinking and perturbing weights, normalization, and regularization toward the initial weights.
After 2,000 tasks the network showed clear loss of plasticity: a freshly initialized one learned task 2,001 far faster.
Origin
Jordan Ash and Ryan Adams (2020) reported that warm-started networks generalize worse than freshly initialized ones, and Clare Lyle and colleagues (2022) described capacity loss in deep RL agents. Shibhansh Dohare, Richard Sutton, A. Rupam Mahmood and colleagues demonstrated loss of plasticity across supervised and reinforcement settings in Nature (2024) and proposed continual backpropagation as a remedy.
Usage note
Distinct from catastrophic forgetting, though both stem from training one network on non-stationary data. The term borrows neuroscience's 'synaptic plasticity'.
Related
- catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on.
- continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch.
- stability–plasticity dilemmastability–plasticity dilemma, n.: The basic tension any learning system faces: it must change enough to learn new things, yet stay stable enough not to lose what it already knows.
- dying ReLUdying ReLU, n.: A training failure in which some of a network's units get stuck permanently switched off, always outputting zero, so they stop learning and become dead weight.
- deep reinforcement learningdeep reinforcement learning, n.: Reinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images.
- weight initializationweight initialization, n.: Choosing the random starting values of a neural network's internal numbers before training. Good starting values keep signals from fading away or blowing up as they pass through many layers.
- catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on.
Sources
- Dohare, S., Hernandez-Garcia, J. F., Lan, Q., Rahman, P., Mahmood, A. R. & Sutton, R. S. (2024). Loss of plasticity in deep continual learning. Nature 632, 768–774.
- Lyle, C., Rowland, M. & Dabney, W. (2022). Understanding and Preventing Capacity Loss in Reinforcement Learning. ICLR 2022.
Mentioned in
catastrophic forgettingcatastrophic forgetting, n.: When a neural network trained on something new gets suddenly and badly worse at what it could already do, because the new training rewrites the internal settings the old skills relied on., continual learningcontinual learning, n.: The ability of an AI to keep learning new things over time from a stream of experience, without forgetting what it already knew and without being retrained from scratch., stability–plasticity dilemmastability–plasticity dilemma, n.: The basic tension any learning system faces: it must change enough to learn new things, yet stay stable enough not to lose what it already knows.