neuroevolution
In plain wordsTraining or designing neural networks by simulated evolution — breeding and mutating whole networks and keeping the best performers — instead of, or alongside, the usual gradient-based training.
The use of evolutionary algorithmsevolutionary computation, n.: Problem-solving by simulated evolution, much as breeders improve crops: keep many candidate answers, let the better ones produce slightly altered offspring, drop the worse, and repeat for many rounds. to optimize neural networksneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain.: their weights, their architectures (topologies), their hyperparameters or learning rules, or several of these at once. Because it needs only a fitness score, neuroevolution suits problems where gradients are unavailable or misleading, such as reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time. with sparse or deceptive rewards. NEAT evolves weights and topology together, starting from minimal networks, using historical markings to align genes for crossover and speciation to protect new structures; HyperNEAT evolves compact patterns that generate large, regular networks. Work since 2017 showed that simple genetic algorithmsgenetic algorithm, n.: A problem-solving method that copies natural evolution: many candidate solutions compete, the best are 'bred' by mixing and randomly tweaking them, and over many generations the solutions improve. and evolution strategiesevolution strategies, n.: A family of evolution-inspired methods for tuning a list of numbers: try many random variations of the current best settings, keep those that work better, and adjust how big the variations are as you go. can train deep networks with millions of parameters competitively on some control tasks, and evolutionary neural architecture searchneural architecture search, n.: Using an automated search, rather than human trial and error, to design the structure of a neural network: which layers to use, how many, and how to connect them. produced image classifiers such as AmoebaNet.
Origin
Experiments evolving network weights date from the late 1980s, for example David Montana and Lawrence Davis (1989). Kenneth Stanley and Risto Miikkulainen's NEAT (2002) made the field prominent, and 2017 work at Uber AI Labs and OpenAI revived it for deep networks.
Related
- evolutionary computationevolutionary computation, n.: Problem-solving by simulated evolution, much as breeders improve crops: keep many candidate answers, let the better ones produce slightly altered offspring, drop the worse, and repeat for many rounds.
- evolution strategiesevolution strategies, n.: A family of evolution-inspired methods for tuning a list of numbers: try many random variations of the current best settings, keep those that work better, and adjust how big the variations are as you go.
- genetic algorithmgenetic algorithm, n.: A problem-solving method that copies natural evolution: many candidate solutions compete, the best are 'bred' by mixing and randomly tweaking them, and over many generations the solutions improve.
- neural architecture searchneural architecture search, n.: Using an automated search, rather than human trial and error, to design the structure of a neural network: which layers to use, how many, and how to connect them.
- quality diversityquality diversity, n.: A family of evolution-inspired search methods that aim to collect many different good solutions — for example, many distinct ways for a robot to walk — rather than a single best one.
- reinforcement learningreinforcement learning, n.: A way of teaching a computer program by trial and error: it tries actions, earns points for good results and penalties for bad ones, and gradually learns which choices pay off over time.
Sources
- Stanley, K. O. & Miikkulainen, R. (2002). Evolving Neural Networks through Augmenting Topologies. Evolutionary Computation 10(2), 99–127.
- Such, F. P. et al. (2017). Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning. arXiv preprint.
Mentioned in
evolutionary computationevolutionary computation, n.: Problem-solving by simulated evolution, much as breeders improve crops: keep many candidate answers, let the better ones produce slightly altered offspring, drop the worse, and repeat for many rounds., genetic algorithmgenetic algorithm, n.: A problem-solving method that copies natural evolution: many candidate solutions compete, the best are 'bred' by mixing and randomly tweaking them, and over many generations the solutions improve., evolution strategiesevolution strategies, n.: A family of evolution-inspired methods for tuning a list of numbers: try many random variations of the current best settings, keep those that work better, and adjust how big the variations are as you go., quality diversityquality diversity, n.: A family of evolution-inspired search methods that aim to collect many different good solutions — for example, many distinct ways for a robot to walk — rather than a single best one.