MyScale's Vector Database Benchmark
Category: Benchmarks & Evaluation
Source: GitHub Repository
Description
MyScale's Vector Database Benchmark is an open-source framework designed to assess the performance of fully-managed vector databases across various search and retrieval tasks. It provides tools and datasets for benchmarking and delivers comparative results for different vector database solutions.
Features
- Benchmarking Framework: Tools to run standardized performance benchmarks on fully-managed vector databases.
- Supported Databases: Includes out-of-the-box support for popular vector databases such as Pinecone, Weaviate, Milvus, Qdrant, and MyScale.
- Workload Types: Measures throughput and cost-performance for both standard vector search and filtered vector search workloads.
- Cost-Performance Analysis: Calculates cost-performance ratio by dividing monthly cost by queries per second (QPS), enabling cost-effectiveness comparisons.
- Dataset Management: Provides scripts and modules for dataset preparation and upload optimization.
- Experiment Management: Organizes and automates experiments for reproducible benchmarking across multiple services.
- Results Visualization: Supports visualization of benchmark results for easier comparison and analysis.
- Open Source: Licensed under Apache-2.0 and extensible for custom needs.
- Hybrid Search Support: Includes support for hybrid search scenarios in MyScale.
- Docker Support: Includes Dockerfiles for containerized environments.
Pricing
Not applicable. This is an open-source tool available for free under the Apache-2.0 license.
Tags
benchmark vector-databases performance retrieval