feature

Machine learning

One of the input variables a model reads for each example: a column in a table (age, income, number of purchases) or a quantity derived from raw data (word counts, the ratio of debt to income); together, an example's features form its feature vector. Statisticians call features predictors, covariates or independent variables. Constructing and transforming them by hand is feature engineeringfeature engineering, n.: Turning raw data into the inputs a model learns from, using knowledge of the problem; for example, converting a timestamp into 'hour of day' and 'is it a weekend'., choosing a subset is feature selectionfeature selection, n.: Choosing which of the available input measurements a model should use and dropping the rest, so the model becomes simpler, faster and often more accurate., putting them on comparable scales is feature scalingfeature scaling, n.: Putting input measurements on comparable scales, so that income in dollars and age in years, say, both span similar ranges and no feature dominates just because of its units., and measuring how much each affects predictions is feature importancefeature importance, n.: A score for how much each input — age, income, location — matters to a model's predictions. It helps people understand, debug and simplify models, but can mislead if read as cause and effect..

Adding 'days since last login' as a feature did more for churn prediction than any change of model.

Deep learning

A learned internal representation: a pattern to which units or layers of a network respond, such as edges and textures in early layers of a convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. and object parts in later ones. A layer's activations form a feature mapfeature map, n.: The grid of numbers produced when one learned pattern detector is scanned across an image, showing where, and how strongly, that pattern appears. or embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance., and using a pretrained network to produce inputs for another model is feature extractionfeature extraction, n.: Turning raw data such as pixels or sound into a short list of telling numbers — computed by hand-designed rules or by an AI system trained earlier — that a new model can learn from or compare.. Deep learning's central claim is that useful features can be learned rather than designed (representation learningrepresentation learning, n.: Letting a computer work out for itself how best to describe its data, such as the edges and shapes in images or the meanings of words, instead of relying on descriptions people design by hand.).

Interpretability

A human-interpretable property that a model represents internally, often modeled as a direction in activation spaceactivation space, n.: The imaginary space of all possible internal states of one layer of a neural network, where each input becomes a point and related meanings tend to share places or directions. and recovered with sparse autoencoderssparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept.; see feature (interpretability)feature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity.. Because of superpositionsuperposition, n.: The way a neural network squeezes more concepts into its internal activity than it has neurons, by letting them overlap — a main reason its insides are hard to read., a network can represent many more such features than it has neurons.

Classical computer vision

A distinctive local image structure, such as a corner, edge or blob, located and described by hand-designed algorithms such as SIFT or HOG for matching, tracking and recognition, the dominant approach before deep learning.