Google Brain
In plain wordsGoogle's deep-learning research team from 2011 to 2023, where the transformer — the design behind today's chatbots — and the TensorFlow software were created.
Google's in-house neural-network research group (2011–2023), which set out to show what very large networks trained on the company's data-center computing could achieve, first in research and then in products. Its 2012 'cat neuron' experiment trained a network without labels on frames from YouTube videos, using 16,000 processor cores, and found units that responded to cats and faces. The team built the DistBelief system and its successor TensorFlowTensorFlow, prop. n.: Google's free software toolkit for building, training and running machine-learning models; it led the field after 2015 and still runs many models inside phones, browsers and company systems.; its researchers co-authored sequence-to-sequence learningsequence-to-sequence model, n.: A model that turns one sequence into another, such as a French sentence into English or speech into text, by first reading the whole input and then writing the output step by step. (2014), knowledge distillationknowledge distillation, n.: Training a smaller or cheaper 'student' AI to copy the behavior of a larger 'teacher' AI, so the student keeps much of the teacher's skill while being faster to run. (2015), sparsely gated mixture of expertsmixture of experts, n.: A way to build very large AI models cheaply: the model contains many specialist sub-networks, and for each word only a few are switched on, picked by a small dispatcher network. layers (2017) and, with Google Research colleagues, Attention Is All You NeedAttention Is All You Need, prop. n.: The 2017 Google research paper that introduced the transformer, the neural-network design behind ChatGPT, Claude, Gemini and nearly every other modern AI language system. (2017), which introduced the transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.. It also put neural machine translation into Google Translate in 2016. Many alumni went on to found or lead other laboratories.
Half of the transformer paper's eight authors did the work at Google Brain, one of them as an intern; the other half were at Google Research.
Origin
Begun in 2011 as a part-time Google X project of Jeff Dean, Greg Corrado and Andrew Ng, then a Stanford professor; it moved into Google Research in 2012. Geoffrey Hinton joined in 2013 when Google acquired his startup DNNresearch. In April 2023 Google merged Brain with DeepMindDeepMind, prop. n.: A London artificial intelligence company, bought by Google in 2014, famous for programs that mastered the game of Go and predicted the shapes of proteins. In 2023 it merged into Google DeepMind. to form Google DeepMindGoogle DeepMind, prop. n.: Google's main artificial intelligence laboratory, formed in 2023 by joining DeepMind and Google Brain. It builds the Gemini models and scientific AI systems such as AlphaFold..
Related
- Google DeepMindGoogle DeepMind, prop. n.: Google's main artificial intelligence laboratory, formed in 2023 by joining DeepMind and Google Brain. It builds the Gemini models and scientific AI systems such as AlphaFold.
- DeepMindDeepMind, prop. n.: A London artificial intelligence company, bought by Google in 2014, famous for programs that mastered the game of Go and predicted the shapes of proteins. In 2023 it merged into Google DeepMind.
- TensorFlowTensorFlow, prop. n.: Google's free software toolkit for building, training and running machine-learning models; it led the field after 2015 and still runs many models inside phones, browsers and company systems.
- transformertransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it.
- Attention Is All You NeedAttention Is All You Need, prop. n.: The 2017 Google research paper that introduced the transformer, the neural-network design behind ChatGPT, Claude, Gemini and nearly every other modern AI language system.
- Andrew NgAndrew Ng, prop. n.: A British-born American computer scientist whose online courses have taught machine learning to millions; he also co-founded Google's deep learning team, Google Brain, and the online-course company Coursera.
- Geoffrey HintonGeoffrey Hinton, prop. n.: A British-Canadian scientist whose decades of persistent work on brain-inspired neural networks laid the foundations of modern AI; often called a 'godfather of AI', he now warns publicly about its dangers.
- Noam ShazeerNoam Shazeer, prop. n.: An American computer scientist who co-invented the transformer and several techniques found in nearly every large language model, co-founded Character.AI, and later returned to Google to co-lead Gemini.
- DeepMindDeepMind, prop. n.: A London artificial intelligence company, bought by Google in 2014, famous for programs that mastered the game of Go and predicted the shapes of proteins. In 2023 it merged into Google DeepMind.
Sources
- Le, Q. V. et al. (2012). Building High-level Features Using Large Scale Unsupervised Learning. Proceedings of ICML 2012.
Mentioned in
Dario AmodeiDario Amodei, prop. n.: An American AI researcher who co-founded Anthropic and is its chief executive, known for early work on scaling and AI safety and for forecasting very rapid AI progress., Google DeepMindGoogle DeepMind, prop. n.: Google's main artificial intelligence laboratory, formed in 2023 by joining DeepMind and Google Brain. It builds the Gemini models and scientific AI systems such as AlphaFold., Andrew NgAndrew Ng, prop. n.: A British-born American computer scientist whose online courses have taught machine learning to millions; he also co-founded Google's deep learning team, Google Brain, and the online-course company Coursera., Chris OlahChris Olah, prop. n.: A Canadian researcher who pioneered ways of looking inside neural networks to see what they have learned, and a co-founder of the AI company Anthropic., DeepMindDeepMind, prop. n.: A London artificial intelligence company, bought by Google in 2014, famous for programs that mastered the game of Go and predicted the shapes of proteins. In 2023 it merged into Google DeepMind., Ilya SutskeverIlya Sutskever, prop. n.: A Russian-born Israeli-Canadian computer scientist who co-built the AlexNet image recognizer, helped found OpenAI and lead its research until 2024, and now heads the startup Safe Superintelligence Inc., Jeff DeanJeff Dean, prop. n.: An American computer scientist who built much of Google's computing infrastructure, co-founded the Google Brain deep-learning team and was Google's chief scientist before leaving in 2026 to co-found a startup., TensorFlowTensorFlow, prop. n.: Google's free software toolkit for building, training and running machine-learning models; it led the field after 2015 and still runs many models inside phones, browsers and company systems., Ashish VaswaniAshish Vaswani, prop. n.: An Indian-born computer scientist listed first on 'Attention Is All You Need', the 2017 paper that introduced the transformer, the design behind today's language models., Jascha Sohl-DicksteinJascha Sohl-Dickstein, prop. n.: An American scientist who in 2015, with colleagues, introduced diffusion models, the technique behind most modern AI image and video generators, by borrowing ideas from physics., Oriol VinyalsOriol Vinyals, prop. n.: A Spanish computer scientist who co-created key neural methods for translation and image captioning, led DeepMind's StarCraft-playing AlphaStar, co-led Google's Gemini models, and in 2026 co-founded the startup Discovery Loop.