Embedding dimensions: what 256 vs 1024 vs 3072 costs in memory, latency and recall

Embedding size is a setting you can change, and it drives the vector store’s memory and search cost. How much retrieval quality do you give up when you shrink it?

Short answer: for a model trained with Matryoshka representation learning (MRL), cutting 1024 dimensions to 256 gave 4x less memory and cost 1.8 points of recall@10 (0.861 to 0.843). Recall@1 fell further, by 6.0 points. Storing the full 1024 dims as int8 gave 4x less memory with no measurable loss.

Setup

• Corpus: the English FastAPI docs. Every h2/h3 section with at least 300 characters of body and a heading of two or more words became one document (first 1,500 characters). That gives 705 documents.
• Queries: the section headings, one per document. A hit means the section’s own document is retrieved. It is an easy, self-labelled test, so read absolute numbers loosely and compare rows against each other.
• Model: mxbai-embed-large-v1, 1024 dimensions, whose model card says it supports Matryoshka representation learning. Inputs were cut at 256 tokens.
• Truncation: keep the first d values, then re-normalize to unit length. Search is exact cosine.

import numpy as np

def truncate(E: np.ndarray, d: int) -> np.ndarray:
    E = E[:, :d]
    return E / np.linalg.norm(E, axis=1, keepdims=True)

Results: recall vs dimension

Dims	Bytes/vector (float32)	Memory vs 1024	Recall@1	Recall@10	MRR
1024	4,096	1x	0.684	0.861	0.752
768	3,072	0.75x	0.679	0.855	0.749
512	2,048	0.5x	0.672	0.854	0.740
256	1,024	0.25x	0.624	0.843	0.705
128	512	0.125x	0.586	0.814	0.670
64	256	0.0625x	0.506	0.779	0.604

Recall@10 is flat down to 512 dims (0.7 points lost) and still within 1.8 points at 256. Recall@1 drops faster: 1.2 points at 512, 6.0 at 256, 9.8 at 128, 17.8 at 64. Truncation mostly shuffles the order of the top few results rather than losing the right document entirely. If you rerank the top 10 or 20 with a cross-encoder, the 256-d recall@10 matters more than recall@1.

Quantization instead of truncation

I also quantized the 1024-d document vectors to int8 (one scale per dimension, taken from the documents) and scored with the dequantized values. Recall@1 was 0.684, recall@10 0.861 and MRR 0.752, identical to float32 at three decimals. That is 1,024 bytes per vector, the same as 256-d float32, with no recall loss here. Caveat: this simulates int8 accuracy in float32. I did not time an int8 index.

Latency and memory at 100k vectors

Dims	Memory (float32)	numpy ms	faiss IndexFlatIP ms
256	102 MB	2.51	6.05
768	307 MB	7.63	16.22
3072	1,229 MB	30.90	69.46

Exact search time scales roughly with dimension. 3072-d is 12.3x slower than 256-d in numpy (30.90 vs 2.51 ms), and memory grows by 12.0x. These vectors are random, since the point is the cost of the size, not recall. 3072 is the default size of OpenAI’s text-embedding-3-large, which, per the OpenAI embeddings guide, can be shortened with a dimensions parameter. I did not call it, so I have no recall number for 3072-d.

As in the brute-force benchmark, plain numpy beat faiss.IndexFlatIP on this 2-thread machine for single queries. At 100k vectors even 3072-d is 30.9 ms. Whether that is too slow depends on your latency budget, not on a rule.

What the numbers mean

• Dimension cuts memory and search time almost linearly. Recall falls slowly, then quickly below roughly 256 dims.
• The first 512 dims of this model hold nearly everything. The curve will differ per model, and only MRL-trained models are designed to be cut this way.
• Dimension is a smaller lever than index size. If you are at 100k chunks, 256 vs 768 saves 205 MB (102 vs 307 MB) and about 5 ms in numpy. At 10M chunks the same ratio is 20 GB, and it starts to matter.

Caveats

• One model, one English technical corpus, 705 heading queries. Real user questions are harder and noisier, and gaps between dimensions may be larger.
• Recall is measured at the section level with a single correct document per query.
• Truncating a model that was not trained with MRL can hurt far more. I did not test one.
• Latency used random vectors on a 2 vCPU machine. Real vectors give the same exact-search cost, but approximate indexes (HNSW) behave differently.
• I did not measure embedding time or API price per dimension.

Rule of thumb

1. Pick an MRL-trained model before you pick a dimension.
2. Start at 512 dims, or at full size with int8, and measure recall@10 on your own queries (see the golden-set post).
3. Go to 256 only if memory pushes you, and keep a reranker if top-1 accuracy matters.
4. Below 128, expect visible losses.

For the basics of embeddings and vector search, see the free guide Embeddings & vector databases. The script and raw output are in the Experiment files box below.