Prompt injection through documents: 12 poisoned chunks, 3 regex detectors, 1 of 8 new attacks caught

Your RAG pipeline pastes retrieved text into the prompt. If someone can put a document in the index, they can put instructions in the prompt. OWASP lists prompt injection as LLM01 in its Top 10 for LLM applications, and indirect injection through retrieved data is the RAG version of it.

Short answer: a keyword filter catches the textbook attack and almost nothing else. A normalizing filter tuned on my 12 fixtures caught all 12, which means little: on 8 payloads I wrote afterwards it caught 1. Treat regex scanning as a cheap tripwire, not a defense.

The setup

Each fixture is a normal docs sentence with an attack appended. Twelve techniques:

direct override, fake SYSTEM: role, paraphrase with no trigger words, markdown-image exfiltration, hidden HTML comment, zero-width characters splitting the words, Cyrillic homoglyphs, base64 payload, Spanish, fake closing </context> tag, tool-call hijack, and an attack split across two chunks.

Three detectors, all run per chunk:

• D1: one regex for “ignore (all) previous instructions”.
• D2: normalizes text first (strip zero-width characters, NFKC, fold a few Cyrillic lookalikes, decode base64 blobs), then applies a broad phrase list, plus checks for external images with query strings and HTML comments.
• D3: the same normalization with a stricter phrase list.

To measure false positives I ran every detector over 4,643 paragraphs of the real FastAPI docs (fetched with experiments/fetch_corpus.sh).

Results on the 12 fixtures

Attack	D1	D2	D3
01 direct override	Y	Y	Y
02 fake system role	-	Y	Y
03 paraphrase, no trigger words	-	Y	Y
04 markdown image exfil	-	Y	Y
05 hidden HTML comment	-	Y	Y
06 zero-width split	-	Y	Y
07 homoglyphs	-	Y	Y
08 base64 payload	-	Y	Y
09 Spanish	-	Y	Y
10 fake closing delimiter	-	Y	Y
11 tool-call hijack	-	Y	-
12 split across 2 chunks	-	Y	-
Detected	1/12	12/12	10/12

D1 catches only the attack everyone has seen. Attack 12 was caught by D2 only because one half contains “rules above no longer”, a phrase I added after seeing the fixture. Joined, the text of attack 12 is flagged by D2 and not by D1 or D3.

The honest part: held-out payloads

I wrote D2 and D3 while looking at these 12 fixtures, so 12/12 is a training score. I then froze the detectors and wrote 8 new payloads (heldout.py): a “your new task is” reframing, “do not mention the sources”, an IMPORTANT: banner with a link, a German override, reversed text, “ignore the above”, a DAN-style role-play and a phishing markdown link.

Detector	Held-out detected
D1	0/8
D2	1/8
D3	1/8

The one hit was “ignore the above”. The other seven were missed, including a German version of an attack D2 catches in Spanish. Blocklists only cover the phrasings their author thought of, and an attacker gets unlimited tries.

False positives

Detector	Flagged benign paragraphs	Rate
D1	0 of 4,643	0.00%
D2	4 of 4,643	0.09%
D3	4 of 4,643	0.09%

All four were HTML comments such as <!-- fmt:off --> and <!-- only-mkdocs -->. Technical docs contain these legitimately, so a blanket “HTML comment = suspicious” rule is cheap here but would be noisier on content that uses comments heavily. Phrases like “you must” or “from now on” never fired on the FastAPI docs, but this is one corpus of technical documentation; support tickets or forum posts would behave differently.

What the numbers mean

• Normalization is what made the structural tricks (zero-width, homoglyphs, base64) detectable. It is cheap and worth doing before any other check.
• Structural signals (external image URLs with query strings, hidden comments, fake role tags) generalized better than phrases, because they do not depend on wording. My held-out set contained only one structural attack, which was missed (a plain markdown link rather than an image), so I can’t claim much more than that.
• Semantic attacks (“your new task is…”, “do not mention the sources”) look like ordinary prose. Regex cannot separate them from content.

Caveats

• No model was called. A payload that is detected might not have worked, and one that is missed might not have worked either.
• 12 and 8 payloads are small samples written by one person. Read the held-out figure as “a blocklist generalizes badly”, not as a precise rate.
• Detection ran per chunk. Attack 12 shows that splitting a payload across chunks defeats per-chunk scanning unless you also scan the assembled prompt.

Rule of thumb

1. Normalize text at ingestion, and scan both at ingestion and on the assembled context.
2. Keep the tripwire, expect it to miss novel wording, and log what it flags.
3. Do not depend on detection. Limit what a successful injection can do: no tools with side effects driven by retrieved text, no rendering of external images or links from model output, and per-user filtering of what can be retrieved (see multi-tenant RAG permissions).
4. Check that answers cite real sources (validating citations).

For the basics of guarding LLM apps, see the free guide on guardrails.

The script, held-out set and raw outputs are in the Experiment files box below.

import re, unicodedata
ZW = re.compile(r"[​-‏⁠]")
def normalize(t: str) -> str:
    return unicodedata.normalize("NFKC", ZW.sub("", t))
IMG = re.compile(r"!\[[^\]]*\]\(https?://[^)]*[?&][^)]*\)")
def suspicious(chunk: str) -> bool:
    n = normalize(chunk)
    return bool(IMG.search(n) or "<!--" in n)