Theano
In plain wordsOne of the first popular free software tools for deep learning, built at a Montreal university lab; development ended in 2017, but it shaped the toolkits that replaced it.
An open-source Python library from the Montreal Institute for Learning Algorithms (MILAMila, prop. n.: Montreal's deep-learning hub, where professors and students from several Quebec universities work side by side. It grew from the lab Yoshua Bengio started in 1993 and is among the world's largest academic AI centers.) that turned array formulas written in Python into optimized machine code. Users built symbolic expression graphs; Theano simplified them algebraically, derived gradients by symbolic automatic differentiationautomatic differentiation, n.: A technique that lets a computer work out exactly how a program's output would change if its inputs changed slightly, by tracking every arithmetic step. It lets AI software compute training updates automatically., and compiled the graph into fast C or CUDA code for CPUs or GPUs. This combination of static computational graphscomputational graph, n.: A map of a calculation drawn as boxes and arrows: each box is a simple operation and each arrow carries a number to the next step. AI software uses it to trace errors backward., automatic gradients and transparent GPU execution made it one of the first practical deep-learning tools and the backend of early high-level libraries such as Lasagne, Blocks and KerasKeras, prop. n.: A free, beginner-friendly Python library for building neural networks from ready-made layers in a few lines of code, while a larger toolkit underneath does the heavy computing., and its design influenced TensorFlowTensorFlow, prop. n.: Google's free software toolkit for building, training and running machine-learning models; it led the field after 2015 and still runs many models inside phones, browsers and company systems.. Long compilation times and the arrival of industry-backed frameworks led MILA to end major development in 2017; community forks continued as Aesara and then PyTensor, used by the PyMC probabilistic-programming library.
The 2014 code still imports Theano, so expect slow graph compilation before the first batch runs.
Origin
Begun around 2007 at the Université de Montréal's LISA laboratory (later MILA) under Yoshua BengioYoshua Bengio, prop. n.: A Canadian computer scientist who helped found deep learning, including ideas behind today's language models, and who has become a leading voice on the risks of advanced AI., with James Bergstra, Olivier Breuleux, Frédéric Bastien and Pascal Lamblin among early developers, and presented at the SciPy conference in 2010. It is named after Theano, a Greek mathematician of the Pythagorean school. Development ended with version 1.0 in November 2017.
Related
- TensorFlowTensorFlow, prop. n.: Google's free software toolkit for building, training and running machine-learning models; it led the field after 2015 and still runs many models inside phones, browsers and company systems.
- PyTorchPyTorch, prop. n.: The most widely used free software toolkit for building and training neural networks, especially in research; programmers write ordinary Python and it handles the calculus and the graphics chips.
- KerasKeras, prop. n.: A free, beginner-friendly Python library for building neural networks from ready-made layers in a few lines of code, while a larger toolkit underneath does the heavy computing.
- automatic differentiationautomatic differentiation, n.: A technique that lets a computer work out exactly how a program's output would change if its inputs changed slightly, by tracking every arithmetic step. It lets AI software compute training updates automatically.
- computational graphcomputational graph, n.: A map of a calculation drawn as boxes and arrows: each box is a simple operation and each arrow carries a number to the next step. AI software uses it to trace errors backward.
- CaffeCaffe, prop. n.: An early free deep-learning toolkit from the University of California, Berkeley, popular for image recognition in the mid-2010s, in which networks were described in simple text files rather than code.
Sources
- Bergstra, J. et al. (2010). Theano: A CPU and GPU Math Compiler in Python. Proceedings of the 9th Python in Science Conference (SciPy 2010).
- Theano Development Team (2016). Theano: A Python framework for fast computation of mathematical expressions. arXiv:1605.02688. arxiv.org/abs/1605.02688
Mentioned in
KerasKeras, prop. n.: A free, beginner-friendly Python library for building neural networks from ready-made layers in a few lines of code, while a larger toolkit underneath does the heavy computing., CaffeCaffe, prop. n.: An early free deep-learning toolkit from the University of California, Berkeley, popular for image recognition in the mid-2010s, in which networks were described in simple text files rather than code.