Alexey Ivakhnenko

Soviet Ukrainian mathematician and engineer at the Institute of Cybernetics in Kyiv who, with Valentin Lapa, built and trained some of the earliest deep, multilayer learning networks. From the mid-1960s he developed what became the group method of data handling (GMDH), which grows a network layer by layer: each unit computes a small polynomial of two inputs from the layer below, units are fitted to training data by regression, and only those that perform best on separate held-out data survive to feed the next layer, so that depth and structure are learned from data; a 1971 paper described a network of eight layers. GMDH was widely used in the Soviet Union for modeling and forecasting. Jürgen SchmidhuberJürgen Schmidhuber, prop. n.: A German computer scientist who co-invented the LSTM, a neural network with memory that powered much of speech recognition and translation in the 2010s, and who argues that early pioneers go under-credited. and others cite Ivakhnenko as the originator of deep learningdeep learning, n.: Machine learning done with neural networks built from many stacked layers: raw data such as pixels, sound or text goes in at one end, and each layer turns what it receives into something more abstract. in the sense of trainable many-layered networks, decades before backpropagationbackpropagation, n.: The method neural networks use to learn from mistakes: it works backward from the error to figure out how much each internal connection contributed, so every connection can be adjusted in the right direction. made deep networks practical; his selection of units on held-out data anticipated the validation setvalidation set, n.: A set of examples kept out of training and used during development to compare model versions and settings, like a practice exam taken before the real one..