Multi-tenant RAG without data leaks: 5 designs, one test suite

If one customer’s question can pull another customer’s chunk into the prompt, you have a data breach, whatever the model does next. How do you test that this can’t happen?

Short answer: plant a canary token in every document that names its owner, then assert that no result ever carries another tenant’s canary. Run it against valid tenants and against bad inputs. In my test, retrieval with no filter and retrieval with a “only use tenant X” rule in the prompt both leaked on 1,800 of 1,800 queries. Per-tenant partitions leaked on none. One design passed every normal query and still leaked on empty or missing tenant ids.

The setup

20 tenants, 200 chunks each, all drawn from the same 400-word vocabulary. That is deliberate: every query has close neighbours in other tenants’ data, which is what real tenants who share a product look like. Each chunk ends with a token like CANARY-t07-0012. A leak is any returned chunk whose canary belongs to someone else.

Each tenant sent 90 queries: 30 using its own content, 30 where an attacker pastes text copied from another tenant’s chunk, and 30 generic ones. That is 1,800 queries per design.

The five designs:

1. No filter. The tenant id is ignored.
2. Prompt-only rule. Retrieval is unfiltered and the prompt says “only use documents of tenant X”. The chunks still reach the model.
3. Post-filter. Fetch the top 50, drop other tenants’ chunks, keep 5. See the filtering post for why this under-returns.
4. Filter skipped when tenant is falsy. if tenant: guards the filter, so "" or None means no filter.
5. Partitioned, fail closed. One matrix per tenant. An unknown tenant raises PermissionError.

The tenant must come from the verified auth token on the server, never from a request body or a model-chosen tool argument. That matters for agents and MCP tools too; see the MCP server post.

Results

Design	Leaked queries	Leaked chunks	Queries with fewer than 5 results
No filter	1800 / 1800	7975	0
Prompt-only rule	1800 / 1800	7975	0
Post-filter top-50	0 / 1800	0	1536
Filter skipped when tenant falsy	0 / 1800	0	0
Partitioned, fail closed	0 / 1800	0	0

The unfiltered designs leaked 7,975 chunks over 1,800 queries, about 4.4 per query out of 5. That is 88.6% of all returned chunks. Not 95% (19 of 20 tenants), because queries still land nearer to their own tenant’s text more often than chance.

Post-filtering never leaked, but 1,536 of 1,800 queries (85.3%) came back with fewer than 5 results. It is safe and incomplete.

The bug the normal suite misses

Design 4 scores 0 leaks, the same as the correct one. The leak only shows up when the tenant id is missing:

Input	Falsy-bug design	Partitioned
tenant=""	5 chunks from 3 tenants	PermissionError
tenant=None	5 chunks from 3 tenants	PermissionError
tenant="t99" (unknown)	0 chunks	PermissionError

A test suite that loops over valid tenants will never find this. Add cases for empty, null, unknown and malformed tenant ids, and expect an error.

Caches leak too

If you cache answers or retrievals by query similarity, the key must include the tenant. I filled a cache with each tenant’s own result, then had every tenant ask the next tenant’s cached query:

Cache key	Hits	Leaked answers
Query only	20 / 20	20 / 20
(tenant, query)	0 / 20	0 / 20

The test, copyable

def test_no_cross_tenant_retrieval(search, tenants, queries_for):
    for t in tenants:
        for q in queries_for(t):            # own text, copied foreign text, generic
            for chunk in search(tenant=t, query=q):
                assert owner_of(chunk) == t, f"{t} got {owner_of(chunk)}'s chunk"

def test_bad_tenant_ids_fail_closed(search):
    for bad in ["", None, "unknown", "t01' OR 1=1", "*"]:
        with pytest.raises(PermissionError):
            search(tenant=bad, query="anything")

Put the canary in the data, not in the test fixture only, so the same assertion can run in staging against real indexes.

Caveats

• Hashed bag-of-words vectors, not a real embedding model. Leak counts for unfiltered retrieval depend on how mixed the tenants are; the point is that a test finds them, not the exact count.
• Per-tenant partitions are the simplest correct design. At large scale you would use your database’s namespaces, row-level security or per-tenant indexes. The same tests apply; I did not test any of them.
• This covers retrieval and caches. It does not cover logs, traces, prompt-injected exfiltration or embedding inversion. OWASP lists tenant isolation under vector and embedding weaknesses; I did not verify that page’s wording during this run.
• Document-level permissions inside a tenant (per-user ACLs) need the same treatment, with canaries per role.

Rule of thumb

Enforce the tenant in the data layer from the authenticated identity, fail closed on anything unexpected, key every cache by tenant, and keep a canary test in CI. A prompt instruction is not an access control. For the performance cost of the filter itself, see the metadata filtering measurements, and for the basics of retrieval see the free retrieval guide.