How much RAM do ten million vectors need?

The first question you ask about a vector database is about result quality. The second one arrives with the bill: why does this server need 64 GB of memory?

The answer is one multiplication. Ten million passages, 1,024-dimension embeddings, four bytes per number: 41 GB, for the vectors alone. Before the index, before metadata, before the engine’s own overhead. It is a calculation to do when you pick an embedding model, not six months later.

This article breaks down what takes up memory in a vector index, then what you can really save, and at what cost. The orders of magnitude come from what I learned building Buddy, whose search runs on Qdrant, but they hold for most engines.

What takes up memory

First, the vectors themselves: number of vectors × dimensions × bytes per dimension. By default, embedding APIs return 32-bit floating point numbers, so four bytes each. A 1,024-dimension vector weighs 4 KB.

Then the index. Most engines use HNSW, a graph where each vector keeps a list of its close neighbours. A search starts from an entry point and hops from neighbour to neighbour towards the query. With the common m = 16 setting, each point keeps around thirty neighbours on the main layer, each stored as a four-byte id. Count roughly 140 bytes per vector, whatever its size.

Then come metadata and each engine’s own overhead, which the calculator leaves out. Qdrant’s documentation suggests a simple rule to account for them: the raw vector size, times 1.5.

Fewer dimensions

An embedding’s dimension is often a choice, not a given. Some recent models are trained so that the first dimensions carry most of the information, a technique called Matryoshka, after the Russian dolls. You can then truncate a vector to 512 or 256 dimensions and lose little. OpenAI and Voyage, for instance, expose a parameter to choose the output dimension.

Halving the dimension halves the vectors’ memory and speeds up every comparison by as much. How little you lose depends on your data, so measure it on an evaluation set of real questions, not on the provider’s benchmark.

Fewer bytes per dimension

Quantization keeps every dimension but stores it with less precision. In float16 you halve the size, with no measurable loss in most cases. In int8, each number is mapped to 256 levels based on the observed range of values: four times less memory, usually for a small loss. It is the setting I turn on almost by default.

Binary quantization goes much further: a single bit per dimension, its sign. Thirty-two times less memory, and comparisons that come down to a few bit operations, extremely fast. But the lost information is real. It works better on high-dimension models, and can hurt on a small 384-dimension one.

Hence rescoring. You search with the binary codes while asking for three or four times more candidates than you need, then recompute the exact score of those candidates with the original vectors. The coarse search finds the right neighbourhood, the exact maths puts it back in order.

Where the originals live

Quantizing does not mean throwing away the original vectors. You move them to disk. Memory keeps the compressed codes and the graph, which every search touches. The SSD keeps the full vectors, read only to rescore a handful of candidates. Disk costs a fraction of RAM, and keeping the originals makes quantization reversible: if recall disappoints, you change the setting without recomputing embeddings.

With Qdrant, that is a few lines of configuration:

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://localhost:6333")

client.create_collection(
    collection_name="passages",
    vectors_config=models.VectorParams(
        size=1024,
        distance=models.Distance.COSINE,
        on_disk=True,  # float32 originals live on disk, read only to rescore
    ),
    quantization_config=models.BinaryQuantization(
        binary=models.BinaryQuantizationConfig(always_ram=True),
    ),
)

hits = client.query_points(
    collection_name="passages",
    query=query_vector,
    limit=10,
    search_params=models.SearchParams(
        # fetch 3x more candidates with the binary codes, then rerank them exactly
        quantization=models.QuantizationSearchParams(rescore=True, oversampling=3.0),
    ),
)

Measure recall, not just memory

Every memory saving is paid for in recall, the share of true nearest neighbours the index finds. Measuring it is simple: for a few hundred questions, compare the tuned index’s top ten with the top ten of an exact search, with no index and no quantization. If recall at 10 drops from 0.98 to 0.95, you still need to check that the right answer stays in the top five end to end: it is the only metric the user feels.

10M × 1,024 dimensions in float32

41 GB

the same in int8

10 GB

the same in binary

1.3 GB

My method

• Do the multiplication before choosing the embedding model.
• Test a reduced dimension on your own questions before paying for the full one.
• Use int8 almost by default, binary only with rescoring, and after measuring.
• Keep the originals on disk: they cost little and keep every option open.
• Remember that the graph has its own weight, which quantization does not shrink.