BigVectorBench

Overview

BigVectorBench is an innovative benchmark suite crafted to thoroughly evaluate the performance of vector databases, born out of the realization that existing benchmarks fall short in assessing the critical capabilities of vector databases, particularly in handling heterogeneous data embeddings and executing compound queries.

Key Features

• Heterogeneous Data Embedding: Evaluates the embedding performance of varied data types (text, images, audio) into a cohesive vector format
• Compound Queries: Tests processing multimodal or single-modal queries with precise constraints
• Multi-modal Workloads: Simulates text-to-image retrieval and other cross-modal search scenarios
• GPU-Accelerated Testing: Evaluates GPU-accelerated databases like Milvus with IVFFlat-GPU
• Docker-based Deployment: Includes Docker-based deployment for testing various vector databases

Use Cases

BigVectorBench is designed to stress-test databases on heterogeneous data and hybrid queries, replacing traditional unimodal vector searches with more realistic compound queries used in production applications.

Availability

The source code and user manual are available on GitHub, with documentation for custom datasets and comprehensive testing scenarios for real-world applications.