hill climbing

A local searchlocal search, n.: A family of problem-solving methods that start with some complete answer, however poor, and keep improving it by small changes, instead of building an answer piece by piece from scratch. algorithm that starts from a candidate solution and repeatedly moves to a neighboring solution that improves the objective — the best neighbor in steepest-ascent hill climbing, the first improving one found in first-choice hill climbing — and stops when no neighbor is better. It uses little memory and often finds good solutions quickly, but it halts at local optima, stalls on plateaus and zigzags slowly along ridges. Standard remedies are random restarts, limited sideways moves, occasional acceptance of worse moves (as in simulated annealingsimulated annealing, n.: An optimization method inspired by slowly cooling metal: early on it sometimes accepts worse solutions, to escape dead ends, and becomes pickier as a 'temperature' setting gradually drops.) and memory of recent moves (as in tabu searchtabu search, n.: A search method that improves an answer by small changes while keeping a short memory of recent moves and forbidding their reversal for a while, so it does not circle back to the same dead ends.). Gradient descentgradient descent, n.: A method for training models by repeatedly nudging every setting a small step in whichever direction most reduces the error, like a hiker who cannot see the valley but keeps stepping wherever the ground tilts down. is its continuous counterpart, descending a cost rather than climbing a value.