Vladimir Vapnik
In plain wordsA Soviet-born American mathematician who built the statistical theory of how machines learn from examples and co-invented support vector machines, a leading method before deep learning.
Soviet-born American mathematician and a founder of statistical learning theory. At the Institute of Control Sciences in Moscow he and Alexey Chervonenkis developed Vapnik–Chervonenkis theory (1968–1974), which bounds how well a model fitted to training data will generalizegeneralization, n.: How well a model copes with new examples it has never seen, rather than just memorizing its training data, like a student who understands the method, not just the answers. in terms of its capacity, measured by the VC dimensionVC dimension, n.: A measure of how flexible a family of prediction rules is: the largest number of points it can label in every possible way. More flexible families need more data to learn reliably., and the principle of structural risk minimization. After moving to AT&T Bell Laboratories around 1990 he co-developed the support vector machinesupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too.: Bernhard Boser, Isabelle Guyon and Vapnik combined maximum-marginmargin, n.: How far, and on which side, an example lies from a classifier's dividing line. A wide gap between the line and the nearest examples suggests the classifier will handle new data well. classifiers with the kernel trickkernel trick, n.: A mathematical shortcut that lets simple straight-line methods find curved patterns, by computing similarities between examples as if the data had been mapped into a far richer space — without ever building that space. (1992), and Corinna Cortes and Vapnik added the soft margin (1995). SVMs were among the most widely used learning methods from the late 1990s until the rise of deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract.. He later worked at NEC Laboratories, Columbia University and Facebook AI Research. His books include 'The Nature of Statistical Learning Theory' (1995). Honors include the IEEE John von Neumann Medal (2017).
Related
- support vector machinesupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too.
- VC dimensionVC dimension, n.: A measure of how flexible a family of prediction rules is: the largest number of points it can label in every possible way. More flexible families need more data to learn reliably.
- kernel trickkernel trick, n.: A mathematical shortcut that lets simple straight-line methods find curved patterns, by computing similarities between examples as if the data had been mapped into a far richer space — without ever building that space.
- marginmargin, n.: How far, and on which side, an example lies from a classifier's dividing line. A wide gap between the line and the nearest examples suggests the classifier will handle new data well.
- PAC learningPAC learning, n.: A mathematical definition of what it means to learn from examples: given enough data, a learner should, with high probability, find a rule that is almost always right.
- generalizationgeneralization, n.: How well a model copes with new examples it has never seen, rather than just memorizing its training data, like a student who understands the method, not just the answers.
Sources
- Cortes, C. & Vapnik, V. (1995). Support-Vector Networks. Machine Learning 20(3): 273–297.
- Vapnik, V. N. (1995). The Nature of Statistical Learning Theory. Springer.
Mentioned in
Bell LabsBell Labs, prop. n.: A famous American industrial research laboratory, long owned by the telephone company AT&T, where the transistor, information theory and key early machine-learning methods were invented., Leslie ValiantLeslie Valiant, prop. n.: A British computer scientist who in 1984 gave the first rigorous definition of what it means for a computer to learn from examples, founding the theory of machine learning.