Perceptrons (book)
In plain wordsA 1969 book by Marvin Minsky and Seymour Papert proving that simple one-layer neural networks cannot learn certain patterns. It is widely blamed for a decade-long slump in neural-network research.
A mathematical monograph by Marvin MinskyMarvin Minsky, prop. n.: An American AI pioneer who helped launch the field and MIT's AI laboratory, co-wrote a famous critique of early neural networks, and pictured the mind as a society of many simple parts. and Seymour Papert (MIT Press, 1969) analyzing what single-layer perceptronsperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong. can and cannot compute. Treating a perceptron as a weighted vote over local features of an image, the authors proved that properties such as parity and connectedness require either features that see the whole input or numbers of features and weights that grow impractically with image size; the two-input case of parity is the XOR problemXOR problem, n.: The classic puzzle of teaching a single artificial neuron the 'one or the other but not both' rule. It cannot be done with one straight dividing line, which showed why networks need hidden layers.. The book acknowledged that networks with hidden layers escape these limits but doubted that any effective training method for them would be found. It appeared as funding and attention were shifting to symbolic AIsymbolic AI, n.: An approach to AI that writes knowledge down as human-readable symbols and rules, such as 'every bird has wings', and reasons by manipulating them step by step instead of learning patterns from data., and neural-network research declined sharply in the 1970s. An expanded 1988 edition added an epilogue responding to Parallel Distributed ProcessingParallel Distributed Processing, prop. n.: A two-volume 1986 book by a group of psychologists and computer scientists that revived interest in brain-like neural networks and made the training method behind modern AI widely known. and backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction..
Perceptrons proved that a single-layer network with local features cannot tell whether a figure is connected.
Origin
Grew out of Minsky and Papert's work at the MIT AI Laboratory in the mid-1960s and was published in 1969. The subtitle, An Introduction to Computational Geometry, reflects the authors' view of the work as mathematics rather than a polemic.
Usage note
How far the book caused the neural-network slump is contested. The authors and some historians say it was misread; others point equally to earlier overclaiming, Frank Rosenblatt's death in 1971 and the lack, until the 1980s, of a practical way to train hidden layers.
Related
- perceptronperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong.
- XOR problemXOR problem, n.: The classic puzzle of teaching a single artificial neuron the 'one or the other but not both' rule. It cannot be done with one straight dividing line, which showed why networks need hidden layers.
- Mark I PerceptronMark I Perceptron, prop. n.: A room-sized machine built around 1960 that learned to recognize simple shapes and letters through a camera made of 400 light sensors. It was one of the first machines built to learn from examples.
- connectionismconnectionism, n.: The view that minds and intelligent machines are best built from huge networks of simple, neuron-like units whose connection strengths are learned, rather than from explicit rules and symbols.
- AI winterAI winter, n.: A slump in which disappointment over AI's unmet promises leads governments and companies to pull back money and attention. Historians count two big ones: one in the 1970s and one starting in the late 1980s.
- multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model.
Sources
- Minsky, M. & Papert, S. (1969). Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: MIT Press. Expanded edition 1988.
Mentioned in
AI winterAI winter, n.: A slump in which disappointment over AI's unmet promises leads governments and companies to pull back money and attention. Historians count two big ones: one in the 1970s and one starting in the late 1980s., neural networkneural network, n.: A kind of computer program that learns a task from examples by tuning thousands to billions of connections between simple number-crunching units, an arrangement loosely inspired by nerve cells in the brain., Frank RosenblattFrank Rosenblatt, prop. n.: An American psychologist who invented the perceptron, the first neural network that learned from examples, and built a machine that learned to recognize simple images., linear separabilitylinear separability, n.: When two groups of points can be perfectly divided by a single straight line or, in more dimensions, a flat surface. Many simple learning methods only fully succeed in this case., Marvin MinskyMarvin Minsky, prop. n.: An American AI pioneer who helped launch the field and MIT's AI laboratory, co-wrote a famous critique of early neural networks, and pictured the mind as a society of many simple parts., perceptronperceptron, n.: One of the first learning machines: a single artificial neuron that sorts inputs into two groups by drawing a straight dividing line, nudging its weights whenever it guesses wrong., XOR problemXOR problem, n.: The classic puzzle of teaching a single artificial neuron the 'one or the other but not both' rule. It cannot be done with one straight dividing line, which showed why networks need hidden layers., Mark I PerceptronMark I Perceptron, prop. n.: A room-sized machine built around 1960 that learned to recognize simple shapes and letters through a camera made of 400 light sensors. It was one of the first machines built to learn from examples., Parallel Distributed ProcessingParallel Distributed Processing, prop. n.: A two-volume 1986 book by a group of psychologists and computer scientists that revived interest in brain-like neural networks and made the training method behind modern AI widely known., Seymour PapertSeymour Papert, prop. n.: A South African-born mathematician who co-wrote 'Perceptrons', a critique of early neural networks, with Marvin Minsky, and invented the children's programming language Logo.