Seppo Linnainmaa

Finnish mathematician and computer scientist who first published the reverse mode of automatic differentiationautomatic differentiation, n.: A technique that lets a computer work out exactly how a program's output would change if its inputs changed slightly, by tracking every arithmetic step. It lets AI software compute training updates automatically., the algorithm at the heart of backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction.. In his 1970 master's thesis at the University of Helsinki, written in Finnish, he analyzed how rounding errors accumulate through a computation and showed how to obtain the derivatives of a program's output with respect to all its intermediate quantities in a single backward sweep through its operations, at a cost proportional to that of the computation itself, giving a FORTRAN implementation; the work appeared in English in the journal BIT in 1976. Reverse-mode differentiation, applied to neural networks by Paul WerbosPaul Werbos, prop. n.: An American scientist whose 1974 doctoral thesis set out backpropagation, the method of passing errors backward that now trains nearly all neural networks. and popularized in 1986 by David RumelhartDavid Rumelhart, prop. n.: An American psychologist who, with Geoffrey Hinton and Ronald Williams, showed in 1986 how to train multilayer neural networks, launching the modern revival of the field., Geoffrey HintonGeoffrey Hinton, prop. n.: A British-Canadian scientist whose decades of persistent work on brain-inspired neural networks laid the foundations of modern AI; often called a 'godfather of AI', he now warns publicly about its dangers. and Ronald Williams, is how every deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract. framework now computes gradients. Because credit for backpropagation is disputed, Jürgen SchmidhuberJürgen Schmidhuber, prop. n.: A German computer scientist who co-invented the LSTM, a neural network with memory that powered much of speech recognition and translation in the 2010s, and who argues that early pioneers go under-credited. and others cite his thesis as its origin.