Boosting Deep Vector Quantization with Progressive Distribution Transformation

A commutative vector quantization method for KV cache compression that reduces FP16 cache size by 87.5% with 2-bit quantization and enables 1-bit quantization, allowing LLaMA-3.1 8B to run with 128K context on a single RTX 4090 GPU.

TKDE 2024 paper on optimizing the number of clusters for billion-scale quantization-based nearest neighbor search, providing methods to determine optimal clustering for quantized vector indexing.

SIGMOD 2024 paper introducing RaBitQ, a quantization method for high-dimensional vectors with provable theoretical error bounds for approximate nearest neighbor search in Euclidean space.

ICML 2024 paper presenting a novel residual quantization approach using implicit neural codebooks for vector compression in high-dimensional similarity search, replacing traditional fixed codebooks with learned representations.

ICDE 2024 paper proposing a routing-guided learned product quantization method that enhances graph-based approximate nearest neighbor search by learning optimal quantization guided by graph routing information.

ICML 2020 paper introducing ScaNN (Scalable Nearest Neighbors), a system for accelerating large-scale vector similarity search using anisotropic vector quantization, combining quantization with asymmetric distance computation for high-performance ANN search.