What one RAG query costs: a token-by-token breakdown (with calculator)
A RAG query with 512-token chunks and top-5 sent 2,339 input tokens: about $1.09 per 1,000 queries on gpt-5-mini. Context is 53% of that bill.
Recovering headings from PDFs for RAG chunking: font size alone found 26%
On 5 arXiv PDFs, 'font bigger than body' recovered 26% of headings. Bold plus a numbered pattern hit 77% precision and 53% recall; two PDFs broke every rule.
Does chunk overlap help? Measured at 0, 10, 20 and 40 percent
On the FastAPI docs, 40% overlap raised BM25 strict hit@5 by 13 to 20 points for fixed chunks, but cost 1.6x the index. 10% overlap gave 2 to 5 points.
Chunk size sweep: 200 to 2,000 characters on real docs
Fixed-size chunks fell from 0.58 to 0.16 hit@1 as size grew from 200 to 2,000 characters. Heading-aware chunks only fell from 0.88 to 0.76 on the FastAPI docs.
RAG chunking, measured: heading-aware chunks doubled hit@1 on the FastAPI docs
I tested 5 chunking strategies on 151 real docs and 773 queries. Fixed-size chunks crossed section boundaries 59% of the time. Heading-aware chunks never did.