Diederik Kingma

Dutch machine-learning researcher who co-introduced two staples of deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract.. With his doctoral adviser Max Welling at the University of Amsterdam he proposed the variational autoencodervariational autoencoder, n.: A neural network that learns to squeeze data into a compact, slightly fuzzy code and rebuild it, arranged so that decoding random codes produces new, plausible examples. (2013), which trains a latent-variable generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them. by maximizing the evidence lower boundevidence lower bound, n.: A quantity that is always less than or equal to how well a model explains its data, but much easier to calculate; pushing it upward is how models such as VAEs learn. and makes this tractable through the reparameterization trickreparameterization trick, n.: A trick that rewrites a random draw as fixed numbers plus separate, outside randomness, so that a neural network can still learn through a step that involves chance.; the paper received the first ICLR Test of Time Award (2024). With Jimmy Ba he introduced AdamAdam, prop. n.: The most widely used method for training neural networks. It smooths the direction of updates over time and automatically sets a separate step size for each setting based on how large its recent changes have been. (2014), an optimizer that adapts each parameter's step size from running averages of its gradients and their squares, which became the default for training neural networks and, as AdamWAdamW, prop. n.: A corrected version of the popular Adam training method that applies its steady pull toward smaller weights separately from the main update. It is the standard choice for training large language models., for large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.. He also co-developed the Glow normalizing flownormalizing flow, n.: A generator built from a chain of reversible steps that reshape simple random noise into realistic data; because every step can be undone, it can also calculate exactly how likely any example is. (2018, with Prafulla Dhariwal) and variational diffusion models (2021). A founding research scientist at OpenAIOpenAI, prop. n.: The American company that makes ChatGPT and the GPT models. It began in 2015 as a nonprofit research lab and is now a for-profit company overseen by a nonprofit foundation., he later worked at Google Brain and Google DeepMind and joined AnthropicAnthropic, prop. n.: An American AI company that makes the Claude chatbot and models. It was founded in 2021 by former OpenAI staff and puts particular emphasis on AI safety research. in 2024.