AnythingLLM

llm-app is an open-source, self-hosted framework for building LLM applications with vector database integration for embedding storage and retrieval. Key features include support for various vector stores and free licensing. Suitable for RAG prototypes, offering self-hosted cost advantages over managed services like Pinecone.

LlamaIndex is a Python data framework library for vector search and embedding retrieval, integrating various ANN indexes like HNSW and FAISS without full database dependencies. Supports quantization, multi-modal embeddings, and advanced query engines in Python/Rust backends. Great for prototyping LLM apps and embedded RAG; more developer-friendly and lighter than Milvus, composable vs hnswlib.

RAG-powered text-to-SQL framework that enables natural language querying of SQL databases using vector search for retrieval of relevant schema, documentation, and example queries.

Techniques for managing limited LLM context windows in RAG systems, including chunk selection, summarization, and iterative retrieval. As context windows fill with retrieved documents, strategies ensure the most relevant information reaches the model while respecting token limits.

Using language models to automatically evaluate RAG system outputs, retrieval quality, and answer correctness. LLM-as-judge provides scalable, consistent evaluation of aspects like faithfulness, relevance, and coherence that are difficult to measure with traditional metrics, enabling rapid iteration on RAG systems.

An advanced RAG chunking strategy that uses LLMs to dynamically determine optimal document splitting based on semantic meaning and content structure. Agentic chunking analyzes document characteristics and adapts the chunking approach per document for superior retrieval accuracy.