MyScale's Vector Database Benchmark
Benchmark results and tools by MyScale aimed at measuring the performance of vector databases in various search and retrieval tasks.
About this tool
MyScale's Vector Database Benchmark
Category: Benchmarks & Evaluation
Source: GitHub Repository
Description
MyScale's Vector Database Benchmark is an open-source framework designed to assess the performance of fully-managed vector databases across various search and retrieval tasks. It provides tools and datasets for benchmarking and delivers comparative results for different vector database solutions.
Features
- Benchmarking Framework: Tools to run standardized performance benchmarks on fully-managed vector databases.
- Supported Databases: Includes out-of-the-box support for popular vector databases such as Pinecone, Weaviate, Milvus, Qdrant, and MyScale.
- Workload Types: Measures throughput and cost-performance for both standard vector search and filtered vector search workloads.
- Cost-Performance Analysis: Calculates cost-performance ratio by dividing monthly cost by queries per second (QPS), enabling cost-effectiveness comparisons.
- Dataset Management: Provides scripts and modules for dataset preparation and upload optimization.
- Experiment Management: Organizes and automates experiments for reproducible benchmarking across multiple services.
- Results Visualization: Supports visualization of benchmark results for easier comparison and analysis.
- Open Source: Licensed under Apache-2.0 and extensible for custom needs.
- Hybrid Search Support: Includes support for hybrid search scenarios in MyScale.
- Docker Support: Includes Dockerfiles for containerized environments.
Pricing
Not applicable. This is an open-source tool available for free under the Apache-2.0 license.
Tags
benchmark vector-databases performance retrieval