neural operator
In plain wordsA neural network that learns how a whole physical system responds, such as how a fluid flows given its starting state, so it can stand in for slow simulations and answer new cases quickly.
A neural network whose inputs and outputs are whole functions rather than fixed-size vectors — for example, one that takes the coefficients, forcing or initial condition of a partial differential equation and returns the solution field. Its layers combine pointwise linear maps with learned integral (kernel) operators, so that, in principle, a trained model does not depend on how the functions are discretized and can be run on coarser or finer grids than it was trained on. In the Fourier neural operator the kernel acts as a learned filter on low-frequency modes of the Fourier transformFourier transform, n.: A way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors., making each layer a global convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.; DeepONet instead combines a 'branch' network that encodes the input function with a 'trunk' network that encodes query locations. Trained on simulation data, neural operators serve as surrogates for numerical solvers that can be orders of magnitude faster, in fluid dynamics, weather and climate, materials and subsurface modeling, though accuracy outside the training distribution and stability over long rollouts remain concerns.
Once trained, the Fourier neural operator predicts the flow field in milliseconds instead of the solver's hours.
Origin
DeepONet was introduced by Lu Lu, Pengzhan Jin and George Em Karniadakis (arXiv, October 2019; Nature Machine Intelligence, 2021), building on a 1995 universal approximation theorem for operators by Tianping Chen and Hong Chen. Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart and Anima Anandkumar popularized the name with graph-kernel 'neural operators' (March 2020) and introduced the Fourier neural operator (arXiv, October 2020; ICLR 2021), reporting speedups of up to three orders of magnitude over traditional solvers.
Related
- physics-informed neural networkphysics-informed neural network, n.: A neural network that is penalized during training whenever its output breaks a known physical law, such as an equation of fluid flow, so it can model a system well even from few measurements.
- Fourier transformFourier transform, n.: A way of breaking a signal — a sound, an image, any pattern over time or space — into the simple waves of different frequencies that add up to it, much as a prism splits white light into colors.
- neural ordinary differential equationneural ordinary differential equation, n.: A neural network that, instead of passing data through a fixed stack of layers, describes how the data changes continuously over 'time' and uses an equation solver to follow that change.
- convolutionconvolution, n.: An operation that slides a small grid of weights across an image or signal, multiplying and adding at each spot, to measure how strongly a pattern appears at every location.
- geometric deep learninggeometric deep learning, n.: An approach to designing neural networks around the shapes and symmetries of the data, whether grids, graphs, surfaces or 3-D structures, so the network respects how that data can be moved or rearranged.
Sources
- Li, Z. et al. (2020). Fourier Neural Operator for Parametric Partial Differential Equations. arXiv:2010.08895; ICLR 2021. arxiv.org/abs/2010.08895
- Lu, L., Jin, P., Pang, G., Zhang, Z. & Karniadakis, G. E. (2021). Learning Nonlinear Operators via DeepONet Based on the Universal Approximation Theorem of Operators. Nature Machine Intelligence, 3, 218–229.
Mentioned in
physics-informed neural networkphysics-informed neural network, n.: A neural network that is penalized during training whenever its output breaks a known physical law, such as an equation of fluid flow, so it can model a system well even from few measurements.