AlexNet: the 2012 neural network that launched the deep learning era

Illustrative image: A convolutional neural network: an input image passes through convolution and pooling layers to a classification output; illustrative of the deep learning approach AlexNet popularised. Image: Irisbox, CC BY 4.0, via Wikimedia Commons.
AlexNet is the convolutional neural network whose victory in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) on 30 September 2012 is widely regarded as the moment deep learning broke into the mainstream of artificial intelligence.
The model was built at the University of Toronto by Alex Krizhevsky, working with fellow graduate student Ilya Sutskever and his PhD supervisor Geoffrey Hinton. Competing as team "SuperVision", their entry classified images into 1,000 object categories with a top-5 error rate of 15.3 per cent - more than 10.8 percentage points better than the runner-up, a margin the field had never seen.
The network held 60 million parameters and 650,000 neurons across eight layers: five convolutional layers, some followed by max-pooling, and three fully connected layers. Because the model could not fit into the 3 GB memory of a single graphics card, it was split across two Nvidia GTX 580 GPUs - cards that cost about US$500 each at release - and trained for five to six days on 1.2 million labelled images. The training machine sat in Krizhevsky's bedroom at his parents' house.
Several design choices made the result possible. AlexNet used the ReLU activation function, which trained far faster than the tanh and sigmoid functions then standard; dropout regularization to fight overfitting; and on-the-fly data augmentation that expanded the training set more than 2,000-fold through random crops and horizontal reflections. For the competition itself the team averaged the predictions of seven separately trained AlexNets. The code, written in CUDA and C++ as cuda-convnet, was released under a BSD licence, and the original 2012 source code was formally released by the Computer History Museum in March 2025.
The win rested on three currents that had matured over the previous decade: Fei-Fei Li's ImageNet dataset, built from 2007 and ultimately holding over 14 million labelled images across 22,000 categories; general-purpose GPU computing through Nvidia's CUDA platform; and improved methods for training deep networks. "That moment was pretty symbolic to the world of AI because three fundamental elements of modern AI converged for the first time," Li said in a 2024 interview. At the 2012 European Conference on Computer Vision, Yann LeCun called the model "an unequivocal turning point in the history of computer vision".
The aftermath reshaped the industry. Hinton, Sutskever and Krizhevsky founded DNNresearch soon afterwards and sold it - and the AlexNet code - to Google. Sutskever went on to co-found OpenAI, and Hinton received the 2024 Nobel Prize in Physics, joking: "Ilya thought we should do it, Alex made it work, and I got the Nobel Prize." Deeper successors - GoogLeNet and VGGNet (2014), ResNet (2015) and EfficientNet (2019) - pushed accuracy far beyond AlexNet's, while compact descendants such as SqueezeNet (2016) and MobileNet (2017) brought similar power to phones. By early 2025 the original paper had been cited more than 198,000 times, making it one of the most influential computer science papers ever published.