clustering
In plain wordsAutomatically sorting things into groups of similar items when nobody has said in advance what the groups should be, like sorting a drawer of odd socks into piles by color and size.
An unsupervised learningunsupervised learning, n.: Machine learning without answer keys: the computer looks at unlabeled data and finds structure on its own, such as groups of similar customers or the main ways in which data vary. task in which an algorithm proposes a grouping of unlabeled items, aiming for clusters whose members lie close together under some chosen distance or similarity measure while different clusters stay well apart. With no labels to define a right answer, the outcome hinges on how the items are represented, which measure is used and what shape of group a given algorithm is built to find. The main families are centroid-based (k-meansk-means clustering, n.: A simple way to split data into a chosen number of groups: guess group centers, assign every point to its nearest center, move each center to the middle of its points, and repeat.), connectivity-based (hierarchical clusteringhierarchical clustering, n.: A way of grouping data that builds a family tree of clusters, joining the most similar items step by step, so you can cut the tree at whatever level of detail you need.), density-based (DBSCANDBSCAN, n.: A clustering method that finds groups as crowded regions of data separated by sparse ones, so it can discover oddly shaped groups and set stray points aside as noise.), distribution-based (Gaussian mixture modelsGaussian mixture model, n.: A model that explains data as coming from several overlapping bell-shaped clouds, working out where each cloud sits, how spread out it is, and how likely each point is to belong to it.) and graph-based (spectral clustering). Methods differ in whether the number of clusters must be fixed in advance, whether each point gets one cluster (hard) or membership probabilities (soft), and whether some points may be left out as noise. Quality is judged with internal indices such as the silhouette scoresilhouette score, n.: A number between −1 and 1 that grades a grouping of data by checking, for each point, whether it sits closer to its own group than to the next-nearest group., by agreement with known labels, or by usefulness for a downstream task.
Clustering the support tickets' embeddings surfaced a group of billing complaints that nobody had tagged.
Origin
Grouping by similarity is as old as biological taxonomy. Quantitative methods appeared in the 1930s: the anthropologists Harold Driver and Alfred Kroeber measured similarities between cultures in 1932, and the psychologists Joseph Zubin (1938) and Robert Tryon took the approach up; Tryon's 1939 monograph Cluster Analysis is an early use of the name. Numerical taxonomy in the 1950s–60s and computer algorithms such as k-means spread it through statistics, pattern recognition and data mining.
Usage note
Not to be confused with classificationclassification, n.: Sorting things into categories — spam or not spam, which animal is in a photo, which language a text is written in — by learning from examples that are already labeled., which learns predefined labels from labeled examples; clustering discovers the groups itself. Clusters are hypotheses rather than facts: most algorithms will return clusters even in data with no real group structure.
Related
- unsupervised learningunsupervised learning, n.: Machine learning without answer keys: the computer looks at unlabeled data and finds structure on its own, such as groups of similar customers or the main ways in which data vary.
- k-means clusteringk-means clustering, n.: A simple way to split data into a chosen number of groups: guess group centers, assign every point to its nearest center, move each center to the middle of its points, and repeat.
- hierarchical clusteringhierarchical clustering, n.: A way of grouping data that builds a family tree of clusters, joining the most similar items step by step, so you can cut the tree at whatever level of detail you need.
- DBSCANDBSCAN, n.: A clustering method that finds groups as crowded regions of data separated by sparse ones, so it can discover oddly shaped groups and set stray points aside as noise.
- Gaussian mixture modelGaussian mixture model, n.: A model that explains data as coming from several overlapping bell-shaped clouds, working out where each cloud sits, how spread out it is, and how likely each point is to belong to it.
- silhouette scoresilhouette score, n.: A number between −1 and 1 that grades a grouping of data by checking, for each point, whether it sits closer to its own group than to the next-nearest group.
- dimensionality reductiondimensionality reduction, n.: Squeezing data that has many measurements per item into far fewer numbers while keeping the patterns that matter, so it is easier to visualize, store or learn from.
- embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance.
- classificationclassification, n.: Sorting things into categories — spam or not spam, which animal is in a photo, which language a text is written in — by learning from examples that are already labeled.
Sources
- Tryon, R. C. (1939). Cluster Analysis. Ann Arbor: Edwards Brothers.
- Jain, A. K. (2010). Data Clustering: 50 Years Beyond K-means. Pattern Recognition Letters 31(8).
Mentioned in
classificationclassification, n.: Sorting things into categories — spam or not spam, which animal is in a photo, which language a text is written in — by learning from examples that are already labeled., association rule learningassociation rule learning, n.: Finding 'if this, then often that' patterns in large records of transactions, such as noticing that shoppers who buy pasta and tomatoes often buy parmesan too., curse of dimensionalitycurse of dimensionality, n.: The problems that pile up when each item is described by very many measurements: even huge datasets end up thinly scattered across the possibilities, and 'near' and 'far' stop meaning much., data miningdata mining, n.: Searching large collections of data with computers to discover useful patterns, such as products often bought together or signs that a transaction may be fraudulent., DBSCANDBSCAN, n.: A clustering method that finds groups as crowded regions of data separated by sparse ones, so it can discover oddly shaped groups and set stray points aside as noise., Gaussian mixture modelGaussian mixture model, n.: A model that explains data as coming from several overlapping bell-shaped clouds, working out where each cloud sits, how spread out it is, and how likely each point is to belong to it., hierarchical clusteringhierarchical clustering, n.: A way of grouping data that builds a family tree of clusters, joining the most similar items step by step, so you can cut the tree at whatever level of detail you need., k-means clusteringk-means clustering, n.: A simple way to split data into a chosen number of groups: guess group centers, assign every point to its nearest center, move each center to the middle of its points, and repeat., latent Dirichlet allocationlatent Dirichlet allocation, n.: A popular topic-finding method that assumes every document is a blend of a few themes and every theme favors certain words, then works backward from the words to find both., MTEBMTEB, prop. n.: A large public test suite that scores AI models which turn text into lists of numbers on many different jobs, such as searching, grouping, classifying and matching translations, so people can compare them on one leaderboard., principal component analysisprincipal component analysis, n.: A technique that finds the few directions in which data vary the most and describes each item by its position along them, compressing many measurements into a handful of summary scores., silhouette scoresilhouette score, n.: A number between −1 and 1 that grades a grouping of data by checking, for each point, whether it sits closer to its own group than to the next-nearest group., speaker diarizationspeaker diarization, n.: Working out who spoke when in a recording, splitting a meeting or phone call into turns labeled Speaker 1, Speaker 2 and so on, without necessarily knowing who the speakers are., TF-IDFTF-IDF, abbr.: A simple score for picking out the words that characterize a document: a word counts for more the more often it appears there, and for less the more of the collection's other documents also use it., topic modeltopic model, n.: A method that reads a large pile of documents and discovers the themes running through them, describing each theme by its typical words and each document by its mix of themes., t-SNEt-SNE, n.: A technique for drawing data that have many measurements per item as a flat map in which similar items land close together, widely used to look for groups in AI models' internal data or in genetic data., UMAPUMAP, n.: A fast technique for turning data with many measurements per item into a flat map that keeps similar items together; often preferred to older methods because it is quicker and copes with more data., unsupervised learningunsupervised learning, n.: Machine learning without answer keys: the computer looks at unlabeled data and finds structure on its own, such as groups of similar customers or the main ways in which data vary., self-organizing mapself-organizing map, n.: A neural network that arranges data on a flat grid so that similar items end up in neighboring cells, producing a map of how the data is organized., spectral clusteringspectral clustering, n.: A clustering method that treats data points as a network linked by similarity and finds natural groups by cutting the network where its links are weakest., Teuvo KohonenTeuvo Kohonen, prop. n.: A Finnish engineer who invented the self-organizing map, a neural network that arranges complex data on a simple grid so that similar items end up next to each other., vector quantizationvector quantization, n.: Compressing data by replacing each item with the nearest entry from a small catalog of representative examples, so that only the catalog number needs to be stored or sent.