UMAP
In plain wordsA fast technique for turning data with many measurements per item into a flat map that keeps similar items together; often preferred to older methods because it is quicker and copes with more data.
A nonlinear dimensionality reductiondimensionality reduction, n.: Squeezing data that has many measurements per item into far fewer numbers while keeping the patterns that matter, so it is easier to visualize, store or learn from. method ('uniform manifold approximation and projection') grounded in Riemannian geometry and algebraic topology. It builds a weighted k-nearest-neighbork-nearest neighbors, n.: Predicting something about a new example by finding the few most similar examples already known and going with their majority answer, or their average. graph of the data, treated as a fuzzy topological representation of the underlying manifold, with distances rescaled locally so that the data look uniformly spread. It then optimizes a low-dimensional layout whose own fuzzy graph matches the original, minimizing a cross-entropy between edge weights by stochastic gradient descent with negative sampling. The main parameters are n_neighbors, which trades local against global structure, and min_dist, which sets how tightly points pack. Compared with t-SNEt-SNE, n.: A technique for drawing data that have many measurements per item as a flat map in which similar items land close together, widely used to look for groups in AI models' internal data or in genetic data., UMAP is typically faster, scales to millions of points, can embed into any number of dimensions and can place new points into an existing map. It is widely used to visualize single-cell genomics data and model embeddingsembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance., and as a preprocessing step before density-based clusteringclustering, n.: Automatically sorting things into groups of similar items when nobody has said in advance what the groups should be, like sorting a drawer of odd socks into piles by color and size..
We projected the sentence embeddings with UMAP at n_neighbors = 30 and colored the points by topic.
Origin
Introduced by Leland McInnes, John Healy and James Melville in an arXiv preprint first posted in February 2018, together with the open-source umap-learn Python library. It spread rapidly, above all in single-cell biology, where it rivaled and often displaced t-SNE as the default visualization.
Usage note
Claims that UMAP preserves global structure better than t-SNE are disputed: Dmitry Kobak and George Linderman (2021) traced much of the difference to initialization rather than to the algorithms themselves. As with t-SNE, distances and cluster densities in the output should be read cautiously.
Related
- t-SNEt-SNE, n.: A technique for drawing data that have many measurements per item as a flat map in which similar items land close together, widely used to look for groups in AI models' internal data or in genetic data.
- dimensionality reductiondimensionality reduction, n.: Squeezing data that has many measurements per item into far fewer numbers while keeping the patterns that matter, so it is easier to visualize, store or learn from.
- manifold hypothesismanifold hypothesis, n.: The idea that real-world data, such as photos of faces, fills only a thin, smoothly curved sliver of the vast space of all possible data — which is why learning from examples works at all.
- embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance.
- k-nearest neighborsk-nearest neighbors, n.: Predicting something about a new example by finding the few most similar examples already known and going with their majority answer, or their average.
- clusteringclustering, n.: Automatically sorting things into groups of similar items when nobody has said in advance what the groups should be, like sorting a drawer of odd socks into piles by color and size.
Sources
- McInnes, L., Healy, J. & Melville, J. (2018). UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv:1802.03426. arxiv.org/abs/1802.03426
- Kobak, D. & Linderman, G. C. (2021). Initialization Is Critical for Preserving Global Data Structure in Both t-SNE and UMAP. Nature Biotechnology 39.
Mentioned in
dimensionality reductiondimensionality reduction, n.: Squeezing data that has many measurements per item into far fewer numbers while keeping the patterns that matter, so it is easier to visualize, store or learn from., t-SNEt-SNE, n.: A technique for drawing data that have many measurements per item as a flat map in which similar items land close together, widely used to look for groups in AI models' internal data or in genetic data., activation spaceactivation space, n.: The imaginary space of all possible internal states of one layer of a neural network, where each input becomes a point and related meanings tend to share places or directions., manifold hypothesismanifold hypothesis, n.: The idea that real-world data, such as photos of faces, fills only a thin, smoothly curved sliver of the vast space of all possible data — which is why learning from examples works at all., self-organizing mapself-organizing map, n.: A neural network that arranges data on a flat grid so that similar items end up in neighboring cells, producing a map of how the data is organized., Teuvo KohonenTeuvo Kohonen, prop. n.: A Finnish engineer who invented the self-organizing map, a neural network that arranges complex data on a simple grid so that similar items end up next to each other.