vector
In plain wordsAn ordered list of numbers, like map coordinates. AI systems turn words, images and users into long lists of numbers so that similar things end up as nearby points.
An element of a vector space; concretely, an ordered list of n numbers (its components) that can be added to other vectors of the same length and multiplied by scalarsscalar, n.: A single number, such as 3.7 or −2, as opposed to a list or grid of numbers. A model's error score and its learning rate are both scalars.. Geometrically a vector is an arrow or a point in n-dimensional space, with a length (its normnorm, n.: A way of measuring the size, or length, of a list of numbers — for example the straight-line distance from the origin to the point the list describes.) and a direction. In machine learning nearly everything is encoded as vectors: input featuresfeature, n.: One piece of information about each example that a model uses, like a house's size or a pixel's color, or a pattern a model has learned to detect inside itself., the activationsactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment. of each layerlayer, n.: One stage in a neural network: a group of artificial neurons that all take the same input, transform it, and pass the result on to the next stage., and embeddingsembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance. of tokens, sentences or images, often with hundreds to thousands of dimensions. Geometric relations between vectors carry meaning: cosine similaritycosine similarity, n.: A score from −1 to 1 for how closely two lists of numbers point the same way, ignoring their size. It is the standard test of whether two texts mean similar things. measures semantic closeness in vector searchvector search, n.: Searching by closeness instead of exact match: the query and all stored items are lists of numbers, and the search returns the items whose numbers are most similar to the query's., and directions in activation spaceactivation space, n.: The imaginary space of all possible internal states of one layer of a neural network, where each input becomes a point and related meanings tend to share places or directions. can encode concepts, as the linear representation hypothesislinear representation hypothesis, n.: The idea that a neural network stores many concepts as simple directions inside its internal activity, so how much of a concept is present can be read off by measuring along its direction. proposes. Software stores a vector as a one-dimensional array; mathematics further distinguishes column vectors from row vectors.
Each chunk of the manual is embedded as a 1,024-dimensional vector and stored in the index.
Origin
William Rowan Hamilton introduced 'vector' (Latin vehere, 'to carry') in his quaternion papers of 1844–1846 for the imaginary part of a quaternion; the word itself had been used in astronomy, as in 'radius vector', since the early 18th century. Josiah Willard Gibbs and Oliver Heaviside built modern vector analysis in the 1880s, and Giuseppe Peano gave abstract axioms for vector spaces in 1888.
Usage note
In AI usage 'vector' often means an embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance. specifically, as in vector databasevector database, n.: A database built to store embeddings — lists of numbers that represent meaning — and to quickly find the stored items most similar to a new one. It is the memory behind many AI search tools.. In physics a vector is defined by how its components change under a change of coordinates; in machine learning it usually just means a one-dimensional array.
Related
- scalarscalar, n.: A single number, such as 3.7 or −2, as opposed to a list or grid of numbers. A model's error score and its learning rate are both scalars.
- matrixmatrix, n.: A rectangular grid of numbers arranged in rows and columns. Most of what a neural network has learned is stored in large grids like this.
- tensortensor, n.: A block of numbers with any number of dimensions: a single number, a list, a grid, or a stack of grids. AI software stores and processes all its data this way.
- embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance.
- dot productdot product, n.: A score of how well two lists of numbers line up, found by multiplying matching entries and adding the products. It is large when the lists point the same way and zero when they are at right angles.
- normnorm, n.: A way of measuring the size, or length, of a list of numbers — for example the straight-line distance from the origin to the point the list describes.
- vector databasevector database, n.: A database built to store embeddings — lists of numbers that represent meaning — and to quickly find the stored items most similar to a new one. It is the memory behind many AI search tools.
- linear algebralinear algebra, n.: The branch of mathematics about lists and grids of numbers and the even, straight-line transformations between them. It is the everyday language in which neural networks are written and computed.
- scalarscalar, n.: A single number, such as 3.7 or −2, as opposed to a list or grid of numbers. A model's error score and its learning rate are both scalars.
Mentioned in
cosine similaritycosine similarity, n.: A score from −1 to 1 for how closely two lists of numbers point the same way, ignoring their size. It is the standard test of whether two texts mean similar things., dimensionalitydimensionality, n.: How many separate numbers are needed to describe something. A point on a map needs two; an AI's description of a single word may use thousands., dot productdot product, n.: A score of how well two lists of numbers line up, found by multiplying matching entries and adding the products. It is large when the lists point the same way and zero when they are at right angles., linear algebralinear algebra, n.: The branch of mathematics about lists and grids of numbers and the even, straight-line transformations between them. It is the everyday language in which neural networks are written and computed., linear transformationlinear transformation, n.: A rule for moving every point in a space that keeps straight lines straight and the origin in place: rotating, stretching, reflecting or squashing everything evenly. Every grid of numbers describes one., matrixmatrix, n.: A rectangular grid of numbers arranged in rows and columns. Most of what a neural network has learned is stored in large grids like this., normnorm, n.: A way of measuring the size, or length, of a list of numbers — for example the straight-line distance from the origin to the point the list describes., scalarscalar, n.: A single number, such as 3.7 or −2, as opposed to a list or grid of numbers. A model's error score and its learning rate are both scalars., tensortensor, n.: A block of numbers with any number of dimensions: a single number, a list, a grid, or a stack of grids. AI software stores and processes all its data this way.