Embedding Dimensionality

Guide to choosing optimal embedding dimensions balancing accuracy, storage costs, and computational requirements, covering Matryoshka embeddings and dimension reduction techniques.

Representation learning approach encoding information at multiple granularities, allowing embeddings to be truncated while maintaining performance. Enables 14x smaller sizes and 5x faster search.

The size of vector embeddings, typically ranging from 128 to 1536 dimensions for text models. Higher dimensions capture more nuanced semantics but require more storage and computation. Modern techniques like Matryoshka embeddings allow flexible dimension selection from a single model.

Training technique enabling flexible embedding dimensions by learning representations where truncated vectors maintain good performance, achieving 75% cost savings when using smaller dimensions.

Combining dense vector embeddings with sparse representations in a single unified model. Captures both semantic meaning (dense) and exact term matching (sparse) for superior retrieval performance.

Retrieval-Augmented Generation extended to handle multiple modalities including text, images, video, and audio. Uses multimodal embeddings like Gemini Embedding 2 or CLIP to enable cross-modal search and generation.