artificial neuron
In plain wordsThe basic building block of a neural network: a tiny calculator that multiplies each incoming number by a weight, adds them up, and passes the total through a simple function to decide its output.
The unit of computation from which neural networksneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain. are assembled: it scores its inputs by a weighted sum plus a biasbias, n.: A word with several AI meanings: an adjustable number added inside a model; a systematic error in estimates; or unfair treatment of some groups of people by an AI system. term and then applies an activation functionactivation function, n.: The simple rule each artificial neuron applies to its total input to decide its output, such as 'keep positive values, turn negatives into zero'. Without it, a deep network would be no smarter than a single layer. to that score, y = φ(w·x + b). The weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself. and bias are the parametersparameter, n.: One of the adjustable numbers inside an AI model that are tuned during training. Large language models have billions of them, and together they store what the model has learned. adjusted during trainingtraining, n.: The process by which an AI model learns: it is shown huge numbers of examples and its internal settings are adjusted, bit by bit, until its answers get better.; the activation supplies the nonlinearity that lets networks of neurons represent more than linear maps. Neurons are arranged in layerslayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage., and a neural network is a composition of many of them. The model is loosely inspired by biological neurons, which integrate synaptic inputs and fire when a threshold is crossed, but it abstracts away spikes, timing, dendritic computation and chemistry; spiking neuronsspiking neural network, n.: A neural network that, like the brain, communicates with brief pulses, or spikes, sent only when a neuron's charge crosses a threshold. On special chips this can save a great deal of energy. restore some of these details. In practice, deep-learning code rarely represents neurons individually: a whole layer of them is computed as one matrix multiplicationmatrix multiplication, n.: The rule for combining two grids of numbers into a new grid by repeatedly multiplying rows by columns. It is the single calculation AI chips spend most of their time doing. followed by an elementwise function.
Each artificial neuron in the hidden layer sees all 784 pixel values but weights them differently.
Origin
Warren McCulloch and Walter Pitts proposed the first formal model in 1943: a binary threshold unit computing logical propositions. Frank Rosenblatt's perceptron (1957–58) added learnable weights, and Bernard Widrow's ADALINE (1960) treated the threshold as a trainable bias weight. Smooth, differentiable activations became standard with backpropagation in the 1980s. Leon Harmon's 1959 note 'Artificial neuron' in Science, describing a transistor circuit, is an early printed use of the phrase.
Usage note
In interpretability, 'neuron' means one coordinate of a layer's activation vector; such neurons often respond to many unrelated concepts (polysemanticitypolysemanticity, n.: When a single artificial neuron responds to several unrelated things — say, bird beaks, bicycle wheels and a particular typeface — so that no single label describes what it does.), which is why researchers analyze featuresfeature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity. instead.
Related
- perceptronperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong.
- McCulloch–Pitts neuronMcCulloch–Pitts neuron, n.: The first mathematical model of a brain cell as a simple on–off switch: it fires if enough of its inputs are on and no blocking input is. Networks of them can carry out logic.
- activation functionactivation function, n.: The simple rule each artificial neuron applies to its total input to decide its output, such as 'keep positive values, turn negatives into zero'. Without it, a deep network would be no smarter than a single layer.
- layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage.
- neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain.
- weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself.
- spiking neural networkspiking neural network, n.: A neural network that, like the brain, communicates with brief pulses, or spikes, sent only when a neuron's charge crosses a threshold. On special chips this can save a great deal of energy.
- spiking neural networkspiking neural network, n.: A neural network that, like the brain, communicates with brief pulses, or spikes, sent only when a neuron's charge crosses a threshold. On special chips this can save a great deal of energy.
Sources
- McCulloch, W. S. & Pitts, W. (1943). A Logical Calculus of the Ideas Immanent in Nervous Activity. Bulletin of Mathematical Biophysics, 5, 115–133.
Mentioned in
neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain., biasbias, n.: A word with several AI meanings: an adjustable number added inside a model; a systematic error in estimates; or unfair treatment of some groups of people by an AI system., Donald HebbDonald Hebb, prop. n.: A Canadian psychologist who proposed in 1949 that connections between brain cells strengthen when the cells are active together, an idea behind many learning rules for neural networks., dot productdot product, n.: A score of how well two lists of numbers line up, found by multiplying matching entries and adding the products. It is large when the lists point the same way and zero when they are at right angles., feature (interpretability)feature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity., Frank RosenblattFrank Rosenblatt, prop. n.: An American psychologist who invented the perceptron, the first neural network that learned from examples, and built a machine that learned to recognize simple images., McCulloch–Pitts neuronMcCulloch–Pitts neuron, n.: The first mathematical model of a brain cell as a simple on–off switch: it fires if enough of its inputs are on and no blocking input is. Networks of them can carry out logic., perceptronperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong., polysemanticitypolysemanticity, n.: When a single artificial neuron responds to several unrelated things — say, bird beaks, bicycle wheels and a particular typeface — so that no single label describes what it does., Warren McCullochWarren McCulloch, prop. n.: An American brain scientist who, with Walter Pitts, wrote the 1943 paper that first described nerve cells as simple logic units, the seed of today's artificial neural networks., weightsweights, n.: The learned numbers inside a neural network that set how strongly each part influences the next. The finished set of weights is, in effect, the trained model itself., connectomeconnectome, n.: A full chart of how a nervous system is wired: which nerve cells, or which brain regions, are linked to which others., spiking neural networkspiking neural network, n.: A neural network that, like the brain, communicates with brief pulses, or spikes, sent only when a neuron's charge crosses a threshold. On special chips this can save a great deal of energy., Walter PittsWalter Pitts, prop. n.: A largely self-taught American logician who, at about 20 and without a degree, co-invented with Warren McCulloch a simple logical model of a brain cell, the starting point of neural networks.