Confident AI

Deep1B Dataset powers vector DB perf testing as a billion-scale benchmark with 96D deep learning embeddings, used in ANN-Benchmarks and Big-ANN for QPS/latency/recall at scale. Key features include realistic neural feature distributions for scalability validation. Vital for selecting prod vector DBs handling billion-vector workloads; dataset core to benchmarks vs VectorDBBench full systems.

GraphRAG-Bench benchmarks vector DB-enhanced GraphRAG vs vanilla RAG on multi-hop queries, measuring perf metrics like QPS/latency for reasoning tasks across domains. Key features include standardized eval for graph vs vector retrieval in 2025 release. Helps select hybrid prod vector DB setups; graph-focused unlike pure ANN-Benchmarks or VectorDBBench.

Massive Text Embedding Benchmark leaderboard covering 58 datasets across 112 languages and 8 embedding tasks. Industry-standard benchmark for comparing text embedding models.

Zero-shot benchmark for embedding model evaluation on 18 diverse datasets with NDCG@10 and Recall@100 metrics correlating to vector DB QPS/latency in production. Features heterogeneous tasks like QA, fact-checking, biomedical retrieval for robust comparisons. Use cases include selecting embeddings for RAG pipelines in vector DBs; complements ANN-Benchmarks indexing focus with retrieval task evaluation, differs from VectorDBBench full-DB tests.

Tests vector DBs on multimodal QPS/latency for heterogeneous embeddings and compound queries including GPU setups. Features Docker-based eval for Milvus etc. on cross-modal retrieval. For selecting multimodal vector DBs; differs from ANN-Benchmarks text-only by adding hybrid workloads vs custom single-DB tests.

Provides QPS/latency/recall benchmarks for ANNS algorithms on billion-point datasets via NeurIPS tools for dataset prep and evaluation. Features scalable testing for extreme throughput and visualization. Key for production vector DBs at scale; extends ANN-Benchmarks with billion-scale tools unlike full-system DB benchmarks.