AlexNet
In plain wordsThe neural network that won a major image-recognition contest by a huge margin in 2012, convincing researchers and companies that deep learning trained on graphics chips was the way forward.
A deep convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. built by Alex KrizhevskyAlex Krizhevsky, prop. n.: A Canadian computer scientist who built AlexNet, the image-recognition network that was trained on gaming graphics chips and whose 2012 contest win launched the deep learning boom., Ilya SutskeverIlya Sutskever, prop. n.: A Russian-born Israeli-Canadian computer scientist who co-built the AlexNet image recognizer, helped found OpenAI and lead its research until 2024, and now heads the startup Safe Superintelligence Inc. and Geoffrey HintonGeoffrey Hinton, prop. n.: A British-Canadian scientist whose decades of persistent work on brain-inspired neural networks laid the foundations of modern AI; often called a 'godfather of AI', he now warns publicly about its dangers. at the University of Toronto that won the 2012 ImageNetImageNet, prop. n.: A huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom. Large Scale Visual Recognition Challenge with a top-5 error of 15.3 percent, against 26.2 percent for the next-best entry. It had five convolutional and three fully connected layers, about 60 million parameters and 650,000 neurons, and was trained for five to six days on two NVIDIA GTX 580 GPUsGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time. using Krizhevsky's cuda-convnet code. Its ingredients — ReLUReLU, abbr.: The most common rule a neuron uses to produce its output: keep the number if it is positive, replace it with zero if it is negative. Simple, fast and surprisingly effective. activations, dropoutdropout, n.: A training trick that randomly switches off a fraction of a network's neurons at each step, so the network cannot lean too heavily on any one of them and learns sturdier patterns., extensive data augmentationdata augmentation, n.: Creating extra training examples by changing existing ones in ways that keep their meaning — a flipped, cropped or recolored photo of a cat still shows a cat — so the model learns more robustly., overlapping pooling and GPU training — were individually known, but their combination at scale proved decisive. The result converted computer vision to deep learning almost overnight and is the conventional starting point of the deep learning revolutiondeep learning revolution, n.: The rapid takeover of AI by many-layered neural networks from about 2012, when they suddenly began beating older methods at recognizing images and speech, leading to today's AI systems..
AlexNet's margin over hand-engineered features ended the argument in computer vision.
Origin
Entered in the 2012 challenge under the team name SuperVision and described in the NIPS 2012 paper 'ImageNet Classification with Deep Convolutional Neural Networks'; the name AlexNet, after Krizhevsky, was applied afterwards. In 2013 Google acquired the authors' three-person startup, DNNresearch.
Usage note
The Computer History Museum, working with Google, released AlexNet's original source code in 2025.
Related
- ImageNetImageNet, prop. n.: A huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom.
- deep learning revolutiondeep learning revolution, n.: The rapid takeover of AI by many-layered neural networks from about 2012, when they suddenly began beating older methods at recognizing images and speech, leading to today's AI systems.
- convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects.
- GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time.
- LeNetLeNet, prop. n.: A family of early image-recognition neural networks built at Bell Labs from 1989 that learned to read handwritten digits. A version read huge numbers of bank checks, and its design underlies today's image models.
- dropoutdropout, n.: A training trick that randomly switches off a fraction of a network's neurons at each step, so the network cannot lean too heavily on any one of them and learns sturdier patterns.
- ReLUReLU, abbr.: The most common rule a neuron uses to produce its output: keep the number if it is positive, replace it with zero if it is negative. Simple, fast and surprisingly effective.
Sources
- Krizhevsky, A., Sutskever, I. & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25 (NIPS 2012).
Mentioned in
computer visioncomputer vision, n.: The part of AI that lets computers make sense of images and video: recognizing faces, reading signs, spotting tumors on scans or judging how far away a car is., deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract., Geoffrey HintonGeoffrey Hinton, prop. n.: A British-Canadian scientist whose decades of persistent work on brain-inspired neural networks laid the foundations of modern AI; often called a 'godfather of AI', he now warns publicly about its dangers., GPUGPU, abbr.: A computer chip first designed to draw video-game graphics that turned out to be ideal for AI, because it can do thousands of simple calculations at the same time., image classificationimage classification, n.: Teaching a computer to say what a picture shows by choosing a label, such as 'cat', 'pneumonia' or 'stop sign', from a fixed list of possibilities., Jensen HuangJensen Huang, prop. n.: The Taiwanese-American co-founder and chief executive of Nvidia, whose graphics chips became the main engine for training and running modern AI., NvidiaNvidia, prop. n.: An American chip company whose graphics processors became the main hardware for training and running AI, making it one of the most valuable companies in the world., convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects., CUDACUDA, prop. n.: Nvidia's software system for programming its graphics chips to do general calculations. Most AI software is built on it, which is a big reason Nvidia chips dominate AI., deep learning revolutiondeep learning revolution, n.: The rapid takeover of AI by many-layered neural networks from about 2012, when they suddenly began beating older methods at recognizing images and speech, leading to today's AI systems., dropoutdropout, n.: A training trick that randomly switches off a fraction of a network's neurons at each step, so the network cannot lean too heavily on any one of them and learns sturdier patterns., Fei-Fei LiFei-Fei Li, prop. n.: A Chinese-American computer scientist whose ImageNet project, millions of hand-labeled photos, gave deep learning the proving ground that launched its boom; she now champions 'human-centered' AI., Ilya SutskeverIlya Sutskever, prop. n.: A Russian-born Israeli-Canadian computer scientist who co-built the AlexNet image recognizer, helped found OpenAI and lead its research until 2024, and now heads the startup Safe Superintelligence Inc., ImageNetImageNet, prop. n.: A huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom., LeNetLeNet, prop. n.: A family of early image-recognition neural networks built at Bell Labs from 1989 that learned to read handwritten digits. A version read huge numbers of bank checks, and its design underlies today's image models., NeurIPSNeurIPS, prop. n.: The largest yearly scientific conference on artificial intelligence and machine learning, held each December, where many of the field's landmark papers were first presented., ReLUReLU, abbr.: The most common rule a neuron uses to produce its output: keep the number if it is positive, replace it with zero if it is negative. Simple, fast and surprisingly effective., Alex KrizhevskyAlex Krizhevsky, prop. n.: A Canadian computer scientist who built AlexNet, the image-recognition network that was trained on gaming graphics chips and whose 2012 contest win launched the deep learning boom., CaffeCaffe, prop. n.: An early free deep-learning toolkit from the University of California, Berkeley, popular for image recognition in the mid-2010s, in which networks were described in simple text files rather than code., GPGPUGPGPU, abbr.: Using a graphics chip for ordinary number-crunching instead of drawing pictures — the trick that, years later, made modern AI training practical., InceptionInception, prop. n.: A family of Google image-recognition networks, the first called GoogLeNet, whose building blocks examine each patch of an image at several scales at once while keeping computation low., Vector InstituteVector Institute, prop. n.: Toronto's publicly and privately funded AI institute, set up in 2017 with Geoffrey Hinton as an adviser, which supports university researchers and helps companies, hospitals and government put their work to use., VGGNetVGGNet, prop. n.: An influential 2014 image-recognition network from Oxford, built by stacking many identical small pattern detectors, which showed that making networks deeper by itself improves accuracy.