spiking neural network
In plain wordsA neural network that, like the brain, communicates with brief pulses, or spikes, sent only when a neuron's charge crosses a threshold. On special chips this can save a great deal of energy.
A neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain. whose units communicate through discrete events in time (spikes) rather than continuous values. Each neuron integrates incoming spikes into a membrane potential that decays over time; when the potential crosses a threshold the neuron emits a spike and resets, as in the widely used leaky integrate-and-fire model. Information can be carried by spike rates, precise timing or both. Because computation happens only when spikes occur, SNNs on event-driven neuromorphic hardwareneuromorphic computing, n.: Computer chips designed to work more like brains, with artificial neurons that send brief spikes only when needed, aiming to do AI tasks with far less energy., such as Intel's Loihi, IBM's TrueNorth or the University of Manchester's SpiNNaker, can be far more energy-efficient for sparse, temporal signals like those from event cameras. Training is difficult because spikes are not differentiable; approaches include surrogate gradients, conversion of trained conventional networks and local, biologically inspired rules such as spike-timing-dependent plasticity. SNNs generally still trail conventional deep networks in accuracy on standard benchmarks.
Converted to a spiking neural network, the keyword spotter ran on the neuromorphic chip at a few milliwatts.
Origin
Spiking neuron models go back to Louis Lapicque's integrate-and-fire model (1907) and Alan Hodgkin and Andrew Huxley's 1952 model of the action potential. Wolfgang Maass's 'Networks of Spiking Neurons: The Third Generation of Neural Network Models' (Neural Networks, 1997) framed SNNs as a computational model distinct from threshold and sigmoidal networks.
Related
- artificial neuronartificial neuron, n.: The basic building block of a neural network: a tiny calculator that multiplies each incoming number by a weight, adds them up, and passes the total through a simple function to decide its output.
- neuromorphic computingneuromorphic computing, n.: Computer chips designed to work more like brains, with artificial neurons that send brief spikes only when needed, aiming to do AI tasks with far less energy.
- reservoir computingreservoir computing, n.: A shortcut way to build a network for time-varying signals: a large, fixed, randomly wired network echoes the input, and only a simple final readout is trained.
- Hebbian learningHebbian learning, n.: The idea that a connection between two brain cells grows stronger when they are active at the same time — the principle behind the slogan 'cells that fire together wire together'.
- edge AIedge AI, n.: Doing AI's computing on or near the devices that collect the data, such as cameras, cars, factory machines, phones or local servers, instead of sending everything off to distant data centers.
- artificial neuronartificial neuron, n.: The basic building block of a neural network: a tiny calculator that multiplies each incoming number by a weight, adds them up, and passes the total through a simple function to decide its output.
Sources
- Maass, W. (1997). Networks of Spiking Neurons: The Third Generation of Neural Network Models. Neural Networks, 10(9), 1659–1671.
Mentioned in
artificial neuronartificial neuron, n.: The basic building block of a neural network: a tiny calculator that multiplies each incoming number by a weight, adds them up, and passes the total through a simple function to decide its output., liquid neural networkliquid neural network, n.: A small, brain-inspired network for time-varying data whose neurons change how quickly they react depending on their input, letting a few neurons handle tasks such as steering a car., neuromorphic computingneuromorphic computing, n.: Computer chips designed to work more like brains, with artificial neurons that send brief spikes only when needed, aiming to do AI tasks with far less energy., reservoir computingreservoir computing, n.: A shortcut way to build a network for time-varying signals: a large, fixed, randomly wired network echoes the input, and only a simple final readout is trained.