GraphRAG-Bench

Confident AI evaluates vector DB-integrated LLM apps with 50+ metrics on faithfulness, relevance, tracking QPS/latency in production traces for RAG performance. Key features include DeepEval-powered scoring, observability dashboards, and quality-aware alerting across datasets. Supports prod vector DB RAG selection via real-world eval; broader than ANN-Benchmarks (indexing) or VectorDBBench (DB perf).

Deep1B Dataset powers vector DB perf testing as a billion-scale benchmark with 96D deep learning embeddings, used in ANN-Benchmarks and Big-ANN for QPS/latency/recall at scale. Key features include realistic neural feature distributions for scalability validation. Vital for selecting prod vector DBs handling billion-vector workloads; dataset core to benchmarks vs VectorDBBench full systems.

Massive Text Embedding Benchmark leaderboard covering 58 datasets across 112 languages and 8 embedding tasks. Industry-standard benchmark for comparing text embedding models.

Zero-shot benchmark for embedding model evaluation on 18 diverse datasets with NDCG@10 and Recall@100 metrics correlating to vector DB QPS/latency in production. Features heterogeneous tasks like QA, fact-checking, biomedical retrieval for robust comparisons. Use cases include selecting embeddings for RAG pipelines in vector DBs; complements ANN-Benchmarks indexing focus with retrieval task evaluation, differs from VectorDBBench full-DB tests.

Tests vector DBs on multimodal QPS/latency for heterogeneous embeddings and compound queries including GPU setups. Features Docker-based eval for Milvus etc. on cross-modal retrieval. For selecting multimodal vector DBs; differs from ANN-Benchmarks text-only by adding hybrid workloads vs custom single-DB tests.

Provides QPS/latency/recall benchmarks for ANNS algorithms on billion-point datasets via NeurIPS tools for dataset prep and evaluation. Features scalable testing for extreme throughput and visualization. Key for production vector DBs at scale; extends ANN-Benchmarks with billion-scale tools unlike full-system DB benchmarks.