An intelligent retrieval technique where an LLM decomposes natural language queries into semantic search components and metadata filters. Enables more precise retrieval by automatically extracting structured filters from unstructured queries.
Loading more......
Self-Querying Retriever uses an LLM to decompose natural language queries into two components: a semantic search query and structured metadata filters. This enables more precise retrieval than vector search alone.
User: "Find recent articles about Python written after 2023"
Standard vector search:
LLM breaks query into:
{"year": {"$gt": 2023}, "language": "Python"}results = vectorstore.search(
query="articles about Python", # Semantic
filter={"year": {"$gt": 2023}} # Structured
)
from langchain.retrievers.self_query.base import SelfQueryRetriever
from langchain.chains.query_constructor.base import AttributeInfo
# Define metadata schema
metadata_field_info = [
AttributeInfo(
name="year",
description="The year the document was published",
type="integer",
),
AttributeInfo(
name="language",
description="Programming language",
type="string",
),
]
retriever = SelfQueryRetriever.from_llm(
llm=llm,
vectorstore=vectorstore,
document_contents="Articles about programming",
metadata_field_info=metadata_field_info,
)
docs = retriever.get_relevant_documents(
"Recent Python articles from 2024"
)
"Movies with Tom Hanks from the 1990s"
→ Semantic: "Tom Hanks movies"
→ Filter: {"year": {"$gte": 1990, "$lt": 2000}}
"Cheap hotels near the beach"
→ Semantic: "hotels near beach"
→ Filter: {"price": {"$lt": 100}, "location": "beach"}
Adds small LLM API cost per query for decomposition.