ScaNN — Accelerating Large-Scale Inference with Anisotropic Vector Quantization

A commutative vector quantization method for KV cache compression that reduces FP16 cache size by 87.5% with 2-bit quantization and enables 1-bit quantization, allowing LLaMA-3.1 8B to run with 128K context on a single RTX 4090 GPU.

SIGKDD 2025 paper proposing a progressive distribution transformation approach for boosting deep vector quantization, improving quantization accuracy by progressively adapting data distributions during training.

TKDE 2024 paper on optimizing the number of clusters for billion-scale quantization-based nearest neighbor search, providing methods to determine optimal clustering for quantized vector indexing.

SIGMOD 2024 paper introducing RaBitQ, a quantization method for high-dimensional vectors with provable theoretical error bounds for approximate nearest neighbor search in Euclidean space.

ICML 2024 paper presenting a novel residual quantization approach using implicit neural codebooks for vector compression in high-dimensional similarity search, replacing traditional fixed codebooks with learned representations.

ICDE 2024 paper proposing a routing-guided learned product quantization method that enhances graph-based approximate nearest neighbor search by learning optimal quantization guided by graph routing information.