# Vector Search with Matryoshka Embeddings

[Matryoshka embeddings](https://huggingface.co/blog/matryoshka) are trained to preserve useful information in the earlier dimensions of a vector. The name "Matryoshka" refers to nesting dolls, where smaller dolls are designed to fit inside larger ones. Similarly, shorter prefixes of a Matryoshka embedding retain meaningful semantic relationships. This allows vector search to use shorter prefixes for more efficient retrieval, with the option to rerank candidates using the full embedding for more accurate ranking.

A Matryoshka-capable model generates a single full embedding, which can then be truncated to different prefix lengths for vector search. For example, from a 768-dimensional embedding, an application can use the first 256, 512, or 768 dimensions depending on its retrieval requirements. Fewer dimensions reduce the amount of vector data processed, while more dimensions improve accuracy, allowing applications to trade off search efficiency against retrieval quality.

![](https://images.contentstack.io/v3/assets/bltac01ee6daa3a1e14/blt4297ff6332df9b97/6aa8ee4d8dd5ee2c06092d9d/Matryoshka_Embeddings-ukSihY.png)

Common use cases include:

* **Speed-sensitive search**: Use a shorter prefix for high query volumes or large collections, where reduced vector processing matters more than exact ranking.
* **Accuracy-sensitive search**: Use more dimensions when ranking accuracy is more important.
* **Multi-stage retrieval**: Use a shorter prefix for initial candidate retrieval and a larger representation for a subsequent ranking stage.

## Example Implementation

The following example demonstrates how to generate a Matryoshka embedding, truncate it to different prefix lengths, and compare vector search results at each dimensionality. It uses the [nomic-ai/nomic-embed-text-v1.5 model](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5) and a 100-document sample of the *Pride and Prejudice* text. For each document and query, it generates a single 768-dimensional embedding and evaluates 64-, 128-, 256-, 512-, and 768-dimensional prefixes.

**Note**: This section shows key code excerpts from the example. The complete implementation is available in the [matryoshka-embeddings-example](https://github.com/singlestore-labs/matryoshka-embeddings-example) repository, which includes:

* Nomic Embed v1.5 model setup
* Sample Pride and Prejudice dataset
* Embedding generation
* SingleStore database setup
* Vector search at multiple dimensionalities
* Search-result comparison

## Generate the Embedding

Generate one 768-dimensional embedding for a document, then truncate it to 64, 128, 256, 512, and 768 dimensions:

```python
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "nomic-ai/nomic-embed-text-v1.5",
    trust_remote_code=True
)

DIMENSIONS = [64, 128, 256, 512, 768]

def generate_document_embedding(text, dimensions):
    full_embedding = model.encode(
        f"search_document: {text}",
        convert_to_numpy=True,
        normalize_embeddings=False
    )

    result = {}
    for dim in dimensions:
        truncated = full_embedding[:dim]
        normalized = truncated / np.linalg.norm(truncated)
        result[dim] = normalized.tolist()

    return result
```

This produces five vectors from the same original embedding. Each vector contains the first N dimensions of the full embedding.

**Note**: This example uses L2 normalization after truncation for simplicity. Nomic Embed v1.5 applies layer normalization before truncation, followed by L2 normalization of the truncated vector, so results may differ slightly from the model's recommended usage.

The same prefix-generation process is applied to query embeddings, using the `search_query`: task prefix.

```python
def generate_query_embedding(text, dimensions):
    full_embedding = model.encode(
        f"search_query: {text}",
        convert_to_numpy=True,
        normalize_embeddings=False
    )

    result = {}
    for dim in dimensions:
        truncated = full_embedding[:dim]
        normalized = truncated / np.linalg.norm(truncated)
        result[dim] = normalized.tolist()

    return result
```

## Store Embeddings at Different Dimensionalities

Create a table called `documents` with separate columns for each dimensionality and vector indexes for ANN search. The `VECTOR INDEX` statements create ANN indexes for each dimensionality.

```python
import pymysql
import json

conn = pymysql.connect(host="127.0.0.1", port=3306, user="root")
cursor = conn.cursor()

cursor.execute("CREATE DATABASE IF NOT EXISTS matryoshka_demo")
cursor.execute("USE matryoshka_demo")

cursor.execute("""
    CREATE TABLE IF NOT EXISTS documents (
        id INT PRIMARY KEY AUTO_INCREMENT,
        text TEXT,
        embedding_64 VECTOR(64, F32) NOT NULL,
        embedding_128 VECTOR(128, F32) NOT NULL,
        embedding_256 VECTOR(256, F32) NOT NULL,
        embedding_512 VECTOR(512, F32) NOT NULL,
        embedding_768 VECTOR(768, F32) NOT NULL,
        VECTOR INDEX (embedding_64) INDEX_OPTIONS '{"metric_type":"DOT_PRODUCT"}',
        VECTOR INDEX (embedding_128) INDEX_OPTIONS '{"metric_type":"DOT_PRODUCT"}',
        VECTOR INDEX (embedding_256) INDEX_OPTIONS '{"metric_type":"DOT_PRODUCT"}',
        VECTOR INDEX (embedding_512) INDEX_OPTIONS '{"metric_type":"DOT_PRODUCT"}',
        VECTOR INDEX (embedding_768) INDEX_OPTIONS '{"metric_type":"DOT_PRODUCT"}'
    )
""")

conn.commit()
```

For more information about vector indexing, refer to [Vector Indexing](https://docs.singlestore.com/db/v9.1/reference/sql-reference/vector-functions/vector-indexing.md).

## Insert Documents

Insert each document and its embeddings into the `documents` table.

```python
cols = ["text"] + [f"embedding_{dim}" for dim in DIMENSIONS]
placeholders = ", ".join(["%s"] * len(cols))

batch_data = []
for text in documents:
    embeddings = generate_document_embedding(text, DIMENSIONS)
    batch_data.append([text] + [json.dumps(embeddings[dim]) for dim in DIMENSIONS])

cursor.executemany(
    f"INSERT INTO documents ({', '.join(cols)}) VALUES ({placeholders})",
    batch_data
)
conn.commit()
```

## Search at a Specific Dimensionality

Search using the 256-dimensional embeddings as an example:

```python
query_embeddings = generate_query_embedding(
    "What are the Bennet family's financial concerns?",
    DIMENSIONS
)

cursor.execute("""
    SELECT id, text,
           DOT_PRODUCT(embedding_256, %s :> VECTOR(256)) AS score
    FROM documents
    ORDER BY score DESC
    LIMIT 5
""", (json.dumps(query_embeddings[256]),))

results = cursor.fetchall()

for i, (doc_id, text, score) in enumerate(results[:3], 1):
    preview = text[:60] + "..." if len(text) > 60 else text
    print(f"  {i}. {preview}")
```

## Example Search Results

Semantic queries are run against the *Pride and Prejudice* dataset using 64, 128, 256, 512, and 768 dimensions. The top-five results from each dimensionality are compared with the baseline results.

For example:

Query: `"What are the Bennet family's financial concerns?"`

| Dimensions | Top-5 overlap with 768d |
| ---------- | ----------------------- |
| 64d        | 40%                     |
| 128d       | 40%                     |
| 256d       | 60%                     |
| 512d       | 80%                     |
| 768d       | 100% (baseline)         |

The overlap measures how many of the top-five results from each dimensionality are also returned by the 768-dimensional baseline. For example, a 40% overlap means that two of the five results are the same. This example shows a trend toward greater overlap with the full embedding as the dimensionality increases.

**Note**: Individual query results may vary with small datasets.

## Choosing a Prefix Length

The example demonstrates the trade-off between dimensionality and retrieval quality. A shorter prefix processes fewer vector dimensions, while a longer prefix retains more of the information in the original embedding. Applications should evaluate several prefix lengths using representative data and queries.

## Storage Considerations

This example stores all five dimensionalities in separate columns to demonstrate the trade-offs at each dimension. Although these vectors all originate from the same full embedding, storing them separately requires additional storage and application logic to maintain multiple vectors.

If you are cost-constrained in production, you can store only a smaller dimension (e.g., 256) without storing the full 768-dimensional embedding. This reduces storage cost at the expense of some recall, which may be acceptable depending on your requirements.

## Limitations

Matryoshka truncation depends on the embedding model being trained to preserve useful information in earlier dimensions. Truncating an embedding from a model that does not support this approach can significantly reduce retrieval quality.

Lower-dimensional prefixes generally retain less information than the full embedding, so shorter prefixes may reduce accuracy. A prefix that performs well for one model or workload may not provide sufficient retrieval quality for another.

In this example, the Nomic Embed v1.5 model is evaluated on a 100-document dataset. The top-five overlap shows how closely the search results for each dimension match those from the 768-dimensional baseline. It is not a recall measurement against manually labeled relevant results.

## Related Topics

* [Working with Vector Data](https://docs.singlestore.com/db/v9.1/developer-resources/functional-extensions/working-with-vector-data.md)
* [Matryoshka Representation Learning Paper](https://arxiv.org/abs/2205.13147)

***

Modified at: September 15, 2026

Source: [/db/v9.1/developer-resources/functional-extensions/vector-search-with-matryoshka-embeddings/](https://docs.singlestore.com/db/v9.1/developer-resources/functional-extensions/vector-search-with-matryoshka-embeddings/)

(An index of the documentation is available at /llms.txt)