Benchmarks/Vector search
Exact vector search vs HNSW, 10k to 1M vectors
When do you actually need a vector database?
3.5 msexact search, 100k × 384-dim, 2 vCPUs
Short answer: Exact numpy search over 100k 384-dim vectors took 3.5 ms on 2 vCPUs, and 62 ms at 1M. HNSW answered in under a millisecond, but its recall depended heavily on the data: 0.995 at ef=64 on clustered vectors, 0.206 on uniform random ones.
- 384-dim
- 768-dim
- 1536-dim
What the numbers say
- Below about 100k chunks, a numpy matrix multiply is fast enough for almost any RAG app.
- Latency grows roughly linearly with vector count and with dimension.
- Measure HNSW recall on your own data before trusting it.
Exact search latency
| 384 | 10,000 | 15 MB | 0.35 ms | 0.78 ms |
| 384 | 100,000 | 154 MB | 3.5 ms | 8.3 ms |
| 384 | 500,000 | 768 MB | 31 ms | 66 ms |
| 384 | 1,000,000 | 1.5 GB | 62 ms | 122 ms |
| 768 | 100,000 | 307 MB | 10.6 ms | 22.6 ms |
| 768 | 500,000 | 1.5 GB | 57 ms | 120 ms |
| 768 | 1,000,000 | 3.1 GB | 114 ms | 240 ms |
| 1536 | 10,000 | 61 MB | 1.2 ms | 3.1 ms |
| 1536 | 100,000 | 614 MB | 20 ms | 51 ms |
| 1536 | 500,000 | 3.1 GB | 107 ms | 248 ms |
HNSW: latency and recall at ef = 32 / 64 / 128
| clustered | 384 | 100k | 23 s | 0.10 ms, recall 0.954 | 0.17 ms, 0.995 | 0.34 ms, 1.000 |
| clustered | 768 | 100k | 48 s | 0.22 ms, 0.878 | 0.36 ms, 0.985 | 0.68 ms, 1.000 |
| uniform | 384 | 100k | 37 s | 0.16 ms, 0.129 | 0.31 ms, 0.206 | 0.55 ms, 0.281 |
| uniform | 768 | 100k | 69 s | 0.34 ms, 0.058 | 0.63 ms, 0.105 | 1.07 ms, 0.172 |
| uniform | 384 | 500k | 279 s | 0.19 ms, 0.034 | 0.35 ms, 0.057 | 0.67 ms, 0.099 |
Rule of thumb
| under 100k chunks | numpy (or pgvector with no index). Store vectors in a .npy file or Postgres. |
| 100k to 1M chunks | exact search is still fine at low QPS. Add HNSW when p95 latency or QPS demands it, and measure recall. |
| over 1M chunks, or many tenants | a proper ANN index (pgvector HNSW, Qdrant, Milvus, OpenSearch), plus filtering and sharding. |
Setup
2 vCPU Intel Xeon @ 2.1 GHz, 7 GB RAM, Python 3.11, numpy 2.4.4, faiss-cpu 1.15.1, hnswlib. Synthetic normalized float32 vectors. Median of 50 (exact) or 200 (HNSW) single queries.
Reproduce it
The complete script is at the end of the write-up.