RaBitQ
RaBitQ is an open-source library implementing the "Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search" method, providing vector quantization and compression techniques designed to improve efficiency and accuracy of ANN search engines and vector databases operating in high-dimensional spaces.
About this tool
RaBitQ
Category: SDKs & Libraries
Slug: rabitq
Source: https://github.com/gaoj0017/RaBitQ
Description
RaBitQ is an open-source implementation of the SIGMOD 2024 method "Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Search." It provides vector quantization and compression techniques aimed at improving the efficiency and accuracy of approximate nearest neighbor (ANN) search engines and vector databases operating in high-dimensional spaces.
Features
- Implementation of the RaBitQ quantization method for high-dimensional vectors.
- Designed for approximate nearest neighbor search in Euclidean space.
- Vector quantization and compression suitable for high-dimensional ANN search engines and vector databases.
- Theoretical error bound for distance estimation between vectors.
- Reference implementation aligned with the SIGMOD 2024 publication.
- Example datasets and preprocessing scripts under
./data/with detailed instructions in./data/README.md. - Index construction components (e.g., IVF-RaBitQ) under
./src/ivf_rabitq.h. - Supporting utilities for vector space handling and fast scan operations (
space.h,fast_scan.h). - Result generation pipeline with outputs stored under
./results/. - Dependency on Eigen for linear algebra operations.
- Technical report (
technical_report.pdf) and published paper citation information for research use. - Companion, more practical library referenced at RaBitQ-Library for extended/production-focused usage.
Technical Requirements
- C++ toolchain capable of building the code in
./src/. - Eigen library installed and available to the build system.
Typical Workflow
- Download and preprocess datasets using the scripts and instructions in
./data/README.md. - Build and run the index construction (e.g., IVF-RaBitQ) from the
srcdirectory. - Run query tests over the indexed datasets.
- Inspect generated metrics and outputs in the
./results/directory.
License
- The repository includes a
LICENSEfile (see GitHub project for exact terms).
Pricing
- Open-source software; no pricing information is provided (use is governed by the repository’s license).
Loading more......
Information
Categories
Tags
Similar Products
6 result(s)vsag is an Alibaba open-source library implementing efficient vector search algorithms, including approximate nearest neighbor search for high-dimensional vectors.
An open-source library for approximate nearest neighbor search in high-dimensional spaces, often used as a backend for vector databases and search engines.
NMSLIB is an efficient similarity search library and toolkit for high-dimensional vector spaces, supporting a variety of indexing algorithms for vector database use cases.
Product Quantization is a compression and indexing technique for vector search that splits vectors into subspaces and quantizes each part separately, allowing vector databases to store large-scale embeddings compactly while supporting efficient ANN search.
DET-LSH is a locality-sensitive hashing scheme that introduces a dynamic encoding tree structure to accelerate approximate nearest neighbor (ANN) search in high-dimensional spaces. While it is a research algorithm rather than a production database, it directly targets the core operation behind vector databases—efficient ANN search over vector embeddings—and is relevant for designing or optimizing vector indexing components within vector database systems.
MRPT (Multi-Resolution Proximity Trees) is an open-source library for fast approximate nearest neighbor search in high-dimensional vector spaces, applicable to vector database backends.