late interaction
In plain wordsA search design that keeps a separate number-list for every word of a document and compares them with the question's words at the last moment — more precise than one list per document, yet still fast.
A retrieval architecture in which query and document are encoded independently, as in a bi-encoderbi-encoder, n.: An AI design that turns a question and a document into number lists separately, so documents can be processed once in advance and compared with any new question in a flash., but each into many vectors — usually one per token or image patch — and relevance is computed afterwards by a cheap interaction between the two sets, typically the sum over query vectors of the maximum similarity to any document vector (MaxSim). It sits between single-vector bi-encoders, which compress each text into one vector, and cross-encoderscross-encoder, n.: An AI model that reads a question and a candidate passage together, side by side, and outputs one relevance score. Accurate but slow, so it is used to reorder a short list., which let every token of both texts interact inside the network (early interaction). Late interaction keeps document encoding offline, captures term-level matches, generalizes well across domains and yields interpretable token alignments, but stores many vectors per document and needs specialized indexes. ColBERTColBERT, prop. n.: A search model that compares a question and a document word by word — matching each question word to its best partner in the document — while still letting documents be processed ahead of time for speed. is the defining model; ColPali applies the idea to images of document pages.
Late interaction beat our single-vector model on long technical queries, but the index was ten times larger.
Origin
Named by Omar Khattab and Matei Zaharia in the 2020 ColBERT paper ('contextualized late interaction'). It builds on the distinction in neural ranking between representation-focused and interaction-focused models drawn by Jiafeng Guo and colleagues in 2016.
Related
- ColBERTColBERT, prop. n.: A search model that compares a question and a document word by word — matching each question word to its best partner in the document — while still letting documents be processed ahead of time for speed.
- bi-encoderbi-encoder, n.: An AI design that turns a question and a document into number lists separately, so documents can be processed once in advance and compared with any new question in a flash.
- cross-encodercross-encoder, n.: An AI model that reads a question and a candidate passage together, side by side, and outputs one relevance score. Accurate but slow, so it is used to reorder a short list.
- dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's.
- rerankingreranking, n.: A second, more careful pass over search results: a fast method grabs a few hundred candidates, then a slower, smarter model reorders them so the best come first.
- vector searchvector search, n.: Searching by closeness instead of exact match: the query and all stored items are lists of numbers, and the search returns the items whose numbers are most similar to the query's.
Sources
- Khattab, O. & Zaharia, M. (2020). ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. SIGIR 2020. arXiv:2004.12832. arxiv.org/abs/2004.12832v1
Mentioned in
cross-encodercross-encoder, n.: An AI model that reads a question and a candidate passage together, side by side, and outputs one relevance score. Accurate but slow, so it is used to reorder a short list., rerankingreranking, n.: A second, more careful pass over search results: a fast method grabs a few hundred candidates, then a slower, smarter model reorders them so the best come first., BEIRBEIR, prop. n.: A collection of 18 different search tests — on topics from medicine to finance to fact-checking — used to see whether a search model works on kinds of data it was never trained on., ColBERTColBERT, prop. n.: A search model that compares a question and a document word by word — matching each question word to its best partner in the document — while still letting documents be processed ahead of time for speed.