radial basis function network
In plain wordsA three-layer network whose hidden neurons each respond to inputs near a particular reference point, like bells placed across the data; the output is a weighted mix of the bells.
A feedforward neural networkfeedforward neural network, n.: A neural network laid out like an assembly line: each stage hands its result only to later stages, never back, so every answer depends on the current input alone and nothing is remembered between inputs. with one hidden layerhidden layer, n.: Any layer of a neural network between the input and the output. Its values are never shown to the user; they are the network's internal working-out. whose units compute a radial function of the distance between the input and a center vector, most often a Gaussian, followed by a linear output layer that takes a weighted sum of these responses. Each hidden unit responds only to inputs near its center, giving local rather than global features, unlike the sigmoidalsigmoid function, n.: An S-shaped curve that squeezes any number into a value between 0 and 1, so the result can be read as a probability or as how far a switch is turned on. units of a multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model.. Training is often two-stage: centers are chosen by k-means clusteringk-means clustering, n.: A simple way to split data into a chosen number of groups: guess group centers, assign every point to its nearest center, move each center to the middle of its points, and repeat. or sampling, then output weights are fitted by linear least squares, which is fast and convex. RBF networks are universal approximators and are closely related to kernel methods, Gaussian processesGaussian process, n.: A way of predicting an unknown curve or surface that gives not just a best guess at each point but an honest error bar, narrow near measured points and wide far from them. and radial-basis interpolation in numerical analysis; they were widely used for function approximation and control in the 1990s.
With 200 Gaussian centers placed by k-means, the RBF network fit the sensor curve in seconds.
Origin
Introduced by David Broomhead and David Lowe in 'Multivariable Functional Interpolation and Adaptive Networks' (Complex Systems, 1988), building on radial-basis interpolation from numerical analysis; John Moody and Christian Darken (1989) and Tomaso Poggio and Federico Girosi (1990) developed fast training methods and the regularization theory.
Related
- multilayer perceptronmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model.
- kernel trickkernel trick, n.: A mathematical shortcut that lets simple straight-line methods find curved patterns, by computing similarities between examples as if the data had been mapped into a far richer space — without ever building that space.
- Gaussian processGaussian process, n.: A way of predicting an unknown curve or surface that gives not just a best guess at each point but an honest error bar, narrow near measured points and wide far from them.
- k-means clusteringk-means clustering, n.: A simple way to split data into a chosen number of groups: guess group centers, assign every point to its nearest center, move each center to the middle of its points, and repeat.
- universal approximation theoremuniversal approximation theorem, n.: A mathematical result showing that a neural network with enough neurons can imitate essentially any smooth input-to-output relationship as closely as you like, though it does not say how to find that network.