RMSProp

An adaptive optimizeroptimizer, n.: The rule a training program uses to turn measured errors into actual changes to a model's internal numbers, deciding the direction and size of each adjustment. that divides each parameter's gradient by the square root of an exponential moving average of its recent squared gradients — the root mean square that gives the method its name — typically with decay 0.9 or 0.99 and a small constant added for stability. Like AdaGradAdaGrad, prop. n.: A training method that gives each of a model's settings its own step size, shrinking steps for settings that have already had many large updates and keeping them larger for rarely updated ones. it equalizes step sizes across parameters whose gradients differ widely in scale, but because old gradients are forgotten the effective learning ratelearning rate, n.: How big a step a model takes each time it adjusts its settings during training. Too large and learning goes haywire; too small and it crawls. does not shrink to zero, which suits the non-stationary, non-convex objectives of deep learning. Variants add momentummomentum, n.: A training trick that lets each update keep part of the previous update's direction, like a ball rolling downhill that builds up speed and coasts over small bumps. or 'center' the estimate by subtracting the squared mean gradient. RMSProp trained several landmark deep reinforcement learningdeep reinforcement learning, n.: Reinforcement learning in which a large neural network does the learning, letting an AI learn tasks such as video games or robot control directly from raw inputs like screen pixels or camera images. systems, including the 2015 deep Q-networkdeep Q-network, n.: A method, made famous by DeepMind in 2013–2015, in which a neural network learned to play dozens of Atari video games from screen pixels and the score alone, many at human level. and A3CA3C, abbr.: A 2016 DeepMind training method in which many copies of an AI agent learn in parallel on separate copies of a task, pooling their updates into one shared neural network.; adding momentum on the gradient and bias correction turns it essentially into AdamAdam, prop. n.: The most widely used method for training neural networks. It smooths the direction of updates over time and automatically sets a separate step size for each setting based on how large its recent changes have been..

The DQN baseline uses RMSProp with a learning rate of 2.5e-4.