State-of-the-art late interaction retrieval model that produces multi-vector token-level representations, enabling efficient and effective passage search with rich contextual understanding.
ColBERT (Contextualized Late Interaction over BERT) introduces a late interaction architecture that independently encodes queries and documents using BERT, then employs a powerful interaction step to model fine-grained similarity.
Unlike single-vector models, ColBERT produces multi-vector representations at the granularity of each token:
ColBERT computes relevance scores via late interaction:
Storage Requirements: Requires an embedding for each token, resulting in significantly more storage than single-vector models. However, token embeddings can be stored at lower dimensions and quantization levels to reduce costs.
ColBERT achieves state-of-the-art results on passage retrieval benchmarks (SIGIR'20, TACL'21, NeurIPS'21, NAACL'22, CIKM'22, ACL'23, EMNLP'23).
Free and open-source, available on GitHub.
Loading more......