The Excellence
Directory Platform Template

Simple open-source LLM evaluation framework similar to Pytest for unit testing LLM outputs. Provides 14+ targeted metrics for RAG and fine-tuning scenarios including hallucination, faithfulness, and contextual relevancy.

A research paper that proposes the first structured roadmap for testing Vector Database Management Systems (VDBMS), analyzing bugs, vulnerabilities, and test challenges unique to vector databases. It provides insights and future directions for improving the reliability and robustness of vector databases.

A comprehensive research roadmap addressing the unique challenges of testing vector database management systems (VDBMS), including approaches for test input generation, oracle definition, and test evaluation tailored to vector databases. The work highlights the complexities of high-dimensional vector data, approximate search semantics, and integration with AI/LLM pipelines, making it a valuable resource for advancing reliability and trustworthiness in vector databases.

An academic paper providing a comprehensive overview of the architecture, empirical defects, and future research roadmap for Vector Database Management Systems (VDBMS). This resource is directly relevant for understanding the current state and challenges in building and testing reliable vector databases.