Jürgen Schmidhuber

German computer scientist whose most influential contribution is long short-term memorylong short-term memory, n.: A type of recurrent network with a separate memory lane and learned 'gates' that decide what to remember, forget and output, letting it keep track of information over long sequences. (LSTM, 1997), introduced with his student Sepp HochreiterSepp Hochreiter, prop. n.: A German computer scientist who explained why early neural networks struggled to learn long sequences and, with Jürgen Schmidhuber, invented the LSTM network that solved the problem., a design that overcame the vanishing gradient problemvanishing gradient problem, n.: A training difficulty in deep or recurrent networks: the learning signal shrinks as it travels back through many layers or time steps, so the early parts of the network barely learn. in recurrent networksrecurrent neural network, n.: A neural network that reads a sequence, such as a sentence or an audio clip, one piece at a time, carrying a running summary forward so that what came earlier can shape how later pieces are understood. and powered speech recognition and translation until the transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.. He became scientific director of the Swiss AI lab IDSIA in Lugano in 1995 and in 2021 also took charge of the AI Initiative at Saudi Arabia's King Abdullah University of Science and Technology (KAUST). His lab also developed connectionist temporal classification (CTC) for speech (2006), highway networks (2015), a precursor of residual connectionsresidual connection, n.: A shortcut in a neural network that adds a block's input straight to its output, so each block only has to learn a correction. It is what makes networks hundreds of layers deep trainable., and fast GPU-trained CNNsconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. that won image-recognition contests in 2011. He proposed forms of meta-learningmeta-learning, n.: Training a system across many different tasks so that it gets better at learning itself, letting it pick up a new task quickly from only a few examples. (1987), artificial curiosity (1990), the neural history compressor (1991) and the self-improving Gödel machineGödel machine, n.: A purely theoretical design, proposed by Jürgen Schmidhuber, for a program allowed to change anything about itself, including how it learns, but only once it has proved that the change is an improvement. (2003). He argues that celebrated later work, including GANsgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people. and linear attentionattention, n.: A way for an AI model to choose what to focus on. For each word it processes, the model scores how relevant every other word is and draws information mostly from the relevant ones., rediscovered his group's ideas, and has disputed the credit given to the 2018 Turing AwardTuring Award, prop. n.: Nicknamed the 'Nobel Prize of computing', the top honor in computer science, awarded each year since 1966 by the Association for Computing Machinery. Several AI pioneers have received it. laureates.