What one RAG query costs: a token-by-token breakdown (with calculator)
A RAG query with 512-token chunks and top-5 sent 2,339 input tokens: about $1.09 per 1,000 queries on gpt-5-mini. Context is 53% of that bill.
Prompt injection through documents: 12 poisoned chunks, 3 regex detectors, 1 of 8 new attacks caught
A regex detector caught 12 of 12 poisoned RAG chunks I wrote it for, but only 1 of 8 fresh ones. Flagged 4 of 4,643 clean FastAPI doc paragraphs.
Recovering headings from PDFs for RAG chunking: font size alone found 26%
On 5 arXiv PDFs, 'font bigger than body' recovered 26% of headings. Bold plus a numbered pattern hit 77% precision and 53% recall; two PDFs broke every rule.
Embedding dimensions: what 256 vs 1024 vs 3072 costs in memory, latency and recall
Truncating a Matryoshka embedding from 1024 to 256 dims cut memory 4x and lost 1.8 points of recall@10 on 705 FastAPI doc queries. int8 lost nothing.
Multi-tenant RAG without data leaks: 5 designs, one test suite
Unfiltered and prompt-only retrieval leaked on 1,800 of 1,800 test queries. Per-tenant partitions leaked on 0. Canary tests, and the falsy-tenant bug they miss.
Streaming LLM answers from FastAPI with SSE: sources first, then tokens
An SSE endpoint sent sources after 52 ms instead of a 1,664 ms JSON wait. One blocking time.sleep in the generator pushed first byte to 3,280 ms with 5 clients.
Make your RAG say 'I don't know': a score threshold, measured
A cosine threshold of 0.40 refused 24 of 30 unanswerable questions and still answered 29 of 30 answerable ones. BM25 scores separated them worse (AUC 0.91 vs 0.99).
Reciprocal Rank Fusion k=60: what the constant does, measured on real BM25 + vector runs
Swept RRF k from 0 to 1000 on 1,119 FastAPI doc queries: k=1 scored 4.1 MRR points above k=60 (0.733 vs 0.692). Past k=100 nothing changes.
Metadata filtering in vector search: post-filter loses results, measured
At 1% selectivity, post-filtering top-100 from HNSW recalled 19% of the true results. In-graph filtering kept 100% but took 17 ms. Measured on 100k vectors.
Does chunk overlap help? Measured at 0, 10, 20 and 40 percent
On the FastAPI docs, 40% overlap raised BM25 strict hit@5 by 13 to 20 points for fixed chunks, but cost 1.6x the index. 10% overlap gave 2 to 5 points.
Chunk size sweep: 200 to 2,000 characters on real docs
Fixed-size chunks fell from 0.58 to 0.16 hit@1 as size grew from 200 to 2,000 characters. Heading-aware chunks only fell from 0.88 to 0.76 on the FastAPI docs.
RAG evals that fail CI: a golden-set harness in about 100 lines of Python
A 30-question golden set with pytest caught 3 of 4 retrieval regressions; a hit@1-only gate would have missed one that cut hit@5 from 0.933 to 0.800.
JEPA vs LLMs: what LeCun's world models actually change for engineers
Analysis: JEPA predicts embeddings, not tokens. My text-chunk stand-in: token-space prediction collapsed 99.4% with 10x less data, embedding-space only 45.8%.
Do you need a vector database? Brute-force search benchmarked from 10k to 1M vectors
Exact numpy search over 100k embeddings takes 3.5 ms on 2 vCPUs. Measured latency, memory and HNSW recall to show when an index actually pays off.
Hybrid search in Python: BM25 + vectors with Reciprocal Rank Fusion, alpha swept 0 to 1
Tested BM25+RRF on 2,713 FastAPI doc chunks: BM25 beat vectors by 49 MRR points on identifiers; vectors beat BM25 by 24 on paraphrases. alpha=0.8 balanced both.
RAG chunking, measured: heading-aware chunks doubled hit@1 on the FastAPI docs
I tested 5 chunking strategies on 151 real docs and 773 queries. Fixed-size chunks crossed section boundaries 59% of the time. Heading-aware chunks never did.
The RAG production checklist: 25 checks before real users see it
25 checks across ingestion, retrieval, generation, evaluation and operations that catch the RAG failures demos hide. Free printable PDF included.
Stop fake citations in RAG: validate them server-side in 25 lines of Python
LLMs will cite passage [7] when you gave them 5. A tested validator that strips invented citations and returns a grounded flag your UI can trust.