Training technique enabling flexible embedding dimensions by learning representations where truncated vectors maintain good performance, achieving 75% cost savings when using smaller dimensions.
Matryoshka Representation Learning (MRL) is a training technique that enables models to produce embeddings where truncated versions maintain good performance. Named after Russian nesting dolls, it allows you to use different embedding sizes from the same model without retraining.
During training, the model learns to encode information at multiple granularities:
75% cost savings when storing 768-dimensional versus 3,072-dimensional vectors:
One model, multiple use cases:
Truncate vectors to smaller dimensions without retraining the model, unlike traditional dimensionality reduction techniques.
Most modern embedding models in 2026 support Matryoshka Representation Learning:
Simply truncate the embedding vector to desired dimension.
Available in many open-source and commercial embedding models.
Loading more......