feature extraction

Deep learning

Using a pretrained network with frozen weights to compute representations (its intermediate or penultimate-layer activationsactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment.) that serve as input features for a new task, usually with only a small classifier or regressor trained on top. This contrasts with fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way., which updates the pretrained weights. Feature extraction is cheap, needs little labeled data and lets one encoder serve many tasks; the quality of the features is often measured with a linear probelinear probe, n.: The simplest kind of probe: a straight-line classifier trained on an AI model's internal activity to test whether some piece of information can be read off directly. or a k-nearest-neighbor evaluation. The approach became standard after studies in 2013–2014 showed that features from an ImageNetImageNet, prop. n.: A huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom.-trained convolutional neural networkconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. transferred well to other recognition problems; today self-supervised and image–text encoders (DINODINO, prop. n.: A family of Meta vision models that learn to understand images from millions of unlabeled photos, by training one network to agree with a slowly updated copy of itself., CLIPCLIP, prop. n.: An OpenAI model that learned to match pictures with their captions from hundreds of millions of web examples, so it can judge how well any piece of text describes any image.) and audio encoders are the usual backbonesbackbone, n.: The main, usually pretrained part of a vision model that turns an image into useful internal features; smaller task-specific parts are attached to it, like limbs to a spine..

We use the frozen encoder as a feature extractor and train a small head for each inspection task.

Classical pattern recognition

The computation of descriptive measurements from raw signals before classification: edges, corners, texture statistics, color histograms, histograms of oriented gradients or SIFTscale-invariant feature transform, n.: A classic method for finding distinctive spots in a photo, such as corners and blobs, and describing them so the same spots can be matched in other photos taken from different distances or angles. descriptors for images; MFCCsMFCC, abbr.: A short list of numbers, usually about 13, that summarizes the shape of a brief slice of sound in a way that mimics human hearing; for decades the standard input to speech recognizers. and other spectral features for audio. In a related sense it means building new variables as combinations of the original ones, as principal component analysisprincipal component analysis, n.: A technique that finds the few directions in which data vary the most and describes each item by its position along them, compressing many measurements into a handful of summary scores. does, whereas feature selectionfeature selection, n.: Choosing which of the available input measurements a model should use and dropping the rest, so the model becomes simpler, faster and often more accurate. only decides which original variables to keep.