Bernard Widrow
In plain wordsAn American electrical engineer who in 1960 built ADALINE, an early trainable neural unit, and invented a learning rule that adjusts weights to reduce errors, still at the heart of adaptive filters.
American electrical engineer at Stanford University whose least mean squares (LMS) algorithm of 1960, devised with his student Marcian 'Ted' Hoff, became the workhorse of adaptive signal processing and an early form of neural-network learning. Also called the Widrow–Hoff or delta rule, LMS adjusts a linear unit's weights in proportion to its error on each example — an early instance of stochastic gradient descentstochastic gradient descent, n.: A faster version of gradient descent that estimates which way is downhill from a small random sample of the training data at each step, instead of checking the whole dataset. — and the pair used it in ADALINEADALINE, prop. n.: An early learning machine from 1960 that adjusted its settings to shrink its errors step by step, the same basic idea used to train today's neural networks., a trainable threshold unit built in hardware with electrochemical memistors; MADALINE networks of many such units followed for pattern recognition and adaptive control. Echo cancellers, noise cancellers and adaptive antennas came to rely on LMS, the subject of his book 'Adaptive Signal Processing' (1985, with Samuel Stearns), and its step-size constant is an ancestor of the modern learning ratelearning rate, n.: How big a step a model takes each time it adjusts its settings during training. Too large and learning goes haywire; too small and it crawls.. Honors include the IEEE Alexander Graham Bell Medal (1986) and the Benjamin Franklin Medal (2001).
Related
- ADALINEADALINE, prop. n.: An early learning machine from 1960 that adjusted its settings to shrink its errors step by step, the same basic idea used to train today's neural networks.
- stochastic gradient descentstochastic gradient descent, n.: A faster version of gradient descent that estimates which way is downhill from a small random sample of the training data at each step, instead of checking the whole dataset.
- learning ratelearning rate, n.: How big a step a model takes each time it adjusts its settings during training. Too large and learning goes haywire; too small and it crawls.
- perceptronperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong.
- online learningonline learning, n.: Learning from data that arrive one piece at a time, updating the model after each new example instead of training once on a fixed collection, as a spam filter does while emails stream in.
- Frank RosenblattFrank Rosenblatt, prop. n.: An American psychologist who invented the perceptron, the first neural network that learned from examples, and built a machine that learned to recognize simple images.
Sources
- Widrow, B. & Hoff, M. E. (1960). Adaptive Switching Circuits. IRE WESCON Convention Record, Part 4: 96–104.
- Widrow, B. & Stearns, S. D. (1985). Adaptive Signal Processing. Prentice-Hall.