Shun-ichi Amari

Japanese mathematician and computational neuroscientist, a professor at the University of Tokyo and later director of the RIKEN Brain Science Institute (2003–2008), whose work anticipated several pillars of neural-network learning. His 1967 theory of adaptive pattern classifiers trained networks with hidden layers by stochastic gradient descentstochastic gradient descent, n.: A faster version of gradient descent that estimates which way is downhill from a small random sample of the training data at each step, instead of checking the whole dataset., nearly two decades before backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction. became widespread (see multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model.). In 1972 he analyzed self-organizing associative-memory networks closely related to the later Hopfield networkHopfield network, n.: A network of on–off units, all connected to one another, that stores patterns as stable states. Given a partial or noisy pattern, it settles into the closest stored one, like recalling a memory from a hint., and he developed mathematical theories of neural fields and of learning dynamics. From the 1980s he founded information geometry, which treats families of probability distributions as curved spaces with the Fisher informationFisher information, n.: A number, or table of numbers, saying how sharply the probability of the data reacts when a model's settings are nudged; the sharper the reaction, the more precisely the data can pin those settings down. as their metric; it yielded the natural gradient (1998), an optimizer that takes steepest-descent steps in that geometry and inspired later second-order optimizationsecond-order optimization, n.: Training methods that use not only the slope of the error but also how that slope is changing — its curvature — to choose better-sized steps. They can need far fewer steps, but each costs more. methods for deep networks. His honors include Japan's Order of Culture (2019) and the Kyoto Prize in Advanced Technology (2025).