feature scaling

The transformation of numeric featuresfeature, n.: One piece of information about each example that a model uses, like a house's size or a pixel's color, or a pattern a model has learned to detect inside itself. to comparable ranges before training. Standardization (z-scoring) subtracts each feature's mean and divides by its standard deviation, giving zero mean and unit variance; min–max scaling maps values linearly onto a fixed interval, usually [0, 1]; robust scaling uses the median and interquartile range to resist outliers; and per-example normalization rescales each vector to unit normnorm, n.: A way of measuring the size, or length, of a list of numbers — for example the straight-line distance from the origin to the point the list describes., common for text vectors and embeddingsembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance.. Scaling matters for methods built on distances or dot products (k-nearest neighborsk-nearest neighbors, n.: Predicting something about a new example by finding the few most similar examples already known and going with their majority answer, or their average., k-meansk-means clustering, n.: A simple way to split data into a chosen number of groups: guess group centers, assign every point to its nearest center, move each center to the middle of its points, and repeat., support vector machinessupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too., PCAprincipal component analysis, n.: A technique that finds the few directions in which data vary the most and describes each item by its position along them, compressing many measurements into a handful of summary scores.), for penalized models whose regularizationregularization, n.: Any technique that stops a model from memorizing its training examples too closely — often by penalizing complexity — so that it works better on new data. treats all coefficients alike, and for gradient descentgradient descent, n.: A method for training models by repeatedly nudging every setting a small step in whichever direction most reduces the error, like a hiker who cannot see the valley but keeps stepping wherever the ground tilts down., which converges faster when inputs have similar scales; tree-based models are largely unaffected. Scaling statistics must be computed on the training settraining set, n.: The examples a machine-learning model learns from, like the worked problems a student practices on, kept separate from the examples later used to check how well it learned. alone and applied unchanged to validation, test and production data. Inside neural networks the analogous job is done by layers such as batch normalizationbatch normalization, n.: A layer that rescales the numbers flowing through a network so that, across each batch of training examples, they keep a steady average and spread. It makes deep networks train faster and more reliably. and layer normalizationlayer normalization, n.: A layer that rescales the numbers inside a network, separately for each example, so they keep a steady average and spread. It, or a simpler variant, is a standard part of the models behind today's chatbots..

The SVM looked useless until we standardized the features; income had been swamping everything else.