reservoir computing
In plain wordsA shortcut way to build a network for time-varying signals: a large, fixed, randomly wired network echoes the input, and only a simple final readout is trained.
An approach to learning from time-varying data in which the input drives a fixed, randomly wired recurrent network (the reservoir) and only a linear readout of the reservoir's states is trained, usually by linear regression. The reservoir projects input histories into a high-dimensional dynamical state; if its dynamics forget old inputs gradually (the 'echo state property', typically encouraged by scaling the recurrent weights' spectral radius to about 1 or below), a linear readout can recover many functions of the recent past. Because training avoids backpropagation through timebackpropagation through time, n.: The way recurrent networks are trained: the network is unrolled into a long chain, one copy per time step, and errors are traced back through every step to adjust the shared weights., it is fast and suits physical substrates: optical, electronic, spintronic and even biological systems have served as reservoirs. Echo state networks (rate-based units) and liquid state machines (spikingspiking neural network, n.: A neural network that, like the brain, communicates with brief pulses, or spikes, sent only when a neuron's charge crosses a threshold. On special chips this can save a great deal of energy. units) are its two classic forms; applications include chaotic time-series prediction and neuromorphicneuromorphic computing, n.: Computer chips designed to work more like brains, with artificial neurons that send brief spikes only when needed, aiming to do AI tasks with far less energy. hardware research.
Only the readout weights are trained, so fitting the echo state network took seconds.
Origin
Introduced independently by Herbert Jaeger as echo state networks (GMD technical report, 2001) and by Wolfgang Maass, Thomas Natschläger and Henry Markram as liquid state machines (Neural Computation, 2002). The umbrella term 'reservoir computing' was established around 2007 by David Verstraeten, Benjamin Schrauwen and colleagues.
Related
- recurrent neural networkrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood.
- spiking neural networkspiking neural network, n.: A neural network that, like the brain, communicates with brief pulses, or spikes, sent only when a neuron's charge crosses a threshold. On special chips this can save a great deal of energy.
- neuromorphic computingneuromorphic computing, n.: Computer chips designed to work more like brains, with artificial neurons that send brief spikes only when needed, aiming to do AI tasks with far less energy.
- time series forecastingtime series forecasting, n.: Predicting future values of something measured over time, such as sales, electricity demand, prices or rainfall, from its past values and related information.
- backpropagation through timebackpropagation through time, n.: The way recurrent networks are trained: the network is unrolled into a long chain, one copy per time step, and errors are traced back through every step to adjust the shared weights.
Mentioned in
recurrent neural networkrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood., spiking neural networkspiking neural network, n.: A neural network that, like the brain, communicates with brief pulses, or spikes, sent only when a neuron's charge crosses a threshold. On special chips this can save a great deal of energy.