Product Quantization (PQ)

A scalable system for approximate nearest neighbor search at web-scale, relevant for implementing and understanding vector database infrastructure for high-dimensional data.

IVF is an indexing technique widely used in vector databases where vectors are clustered into inverted lists (partitions), enabling efficient Approximate Nearest Neighbor search by probing only a subset of relevant partitions at query time.

Ball-tree is a binary tree data structure used for organizing points in a multi-dimensional space, particularly useful in vector databases for nearest neighbor search. It partitions data points into hyperspheres (balls), enabling efficient search and scalability in high-dimensional vector spaces.

Online Product Quantization (O-PQ) is a variant of product quantization designed to support dynamic or streaming data. It enables adaptive updating of quantization codebooks and codes in real-time, making it suitable for vector databases that handle evolving datasets.

Optimized Product Quantization (OPQ) enhances Product Quantization by optimizing space decomposition and codebooks, leading to lower quantization distortion and higher accuracy in vector search. OPQ is widely used in advanced vector databases for improving recall and search quality.

Spectral Hashing is a method for approximate nearest neighbor search that uses spectral graph theory to generate compact binary codes, often applied in vector databases to enhance retrieval efficiency on large-scale, high-dimensional data.