Written by: Chris Fonnesbeck, PyMC
Theano was an open-source Python library designed to optimize and evaluate mathematical expressions that involve multi-dimensional arrays (i.e. tensors). It was developed by a team of researchers at the Montreal Institute for Learning Algorithms (MILA), led by Yoshua Bengio, in collaboration with the University of Montreal, and was first released in 2007. The name “Theano” was inspired by the ancient Greek philosopher and mathematician, Theano of Croton, who was known for her contributions to mathematics and science. The choice of name reflected the library’s focus on mathematical optimization and its ambition to advance the field of deep learning.
The development of Theano was driven by the need for efficient algorithms that could handle large amounts of data for deep learning applications. At the time, the field of deep learning was still in its infancy, and researchers were struggling with the computational limitations of existing frameworks. Theano was created to address these limitations by providing a more efficient and flexible computational backend for deep learning models. Its popularity was due in part to its ability to run on both CPUs and GPUs, making it a versatile tool for developers who need to work with large datasets and complex models. Theano also provided a high-level programming interface that makes it easy for developers to define and train deep learning models.
One of the key features of Theano was its symbolic differentiation engine, which allowed it to compute gradients of complex functions efficiently. Critically, this makes it possible to optimize the parameters of deep learning models using gradient descent. In the early part of the last decade, developers on the PyMC team recognized the utility of symbolic differentiation for powering a newly-developed method for Bayesian statistical inference, namely gradient based Markov chain Monte Carlo, and accordingly built PyMC version 3 around Theano. This vastly improved the performance of models in PyMC, allowing it to fit a wider variety of models using the latest methods.
Despite its importance, Theano was officially discontinued in September 2017. The developers cited the emergence of newer and more user-friendly machine learning libraries, such as TensorFlow and PyTorch, as the reason for discontinuing the project. Shortly thereafter, a fork of Theano was adapted by a team led by Brandon Willard to become a general library for building and evaluating tensor models in the context of probabilistic programming. Importantly, it allowed such models to be compiled to a variety of backends, including Jax and Numba. In 2020, this project was released as the Aesara library and was later forked by the PyMC development team as PyTensor to continue as the computational backend of PyMC.
Although Theano is no longer being actively developed or maintained, its legacy lives on through its contributions to the fields of machine learning and probabilistic programming, and the many research projects it has enabled.
