Benchmarks/Chunking

Chunk size sweep: 200 to 2,000 characters

How big should RAG chunks be?

0.877best strict hit@1 (heading-aware, 200 chars)

Short answer: How you cut matters more than how big. Fixed-size chunks fell from 0.583 to 0.161 strict hit@1 as they grew from 200 to 2,000 characters. Heading-aware chunks only fell from 0.877 to 0.757.

Tested 2 October 2026 · Read the full write-up

Strict hit@1 by chunk size↑ higher is better
0.000.250.500.751.00strict hit@12004006008001,2001,6002,000chunk size (characters)Fixed, 200: 0.583Fixed, 400: 0.503Fixed, 600: 0.447Fixed, 800: 0.391Fixed, 1,200: 0.292Fixed, 1,600: 0.197Fixed, 2,000: 0.161Heading-aware, 200: 0.877Heading-aware, 400: 0.822Heading-aware, 600: 0.799Heading-aware, 800: 0.785Heading-aware, 1,200: 0.763Heading-aware, 1,600: 0.761Heading-aware, 2,000: 0.757
  • Fixed
  • Heading-aware
Chunks that cross a section boundary↓ lower is better
0255075100% of chunks2004006008001,2001,6002,000chunk size (characters)Fixed, 200: 22.5Fixed, 400: 38.9Fixed, 600: 50.9Fixed, 800: 59.8Fixed, 1,200: 70.8Fixed, 1,600: 76.8Fixed, 2,000: 81.3Heading-aware, 200: 0Heading-aware, 400: 0Heading-aware, 600: 0Heading-aware, 800: 0Heading-aware, 1,200: 0Heading-aware, 1,600: 0Heading-aware, 2,000: 0
  • Fixed
  • Heading-aware

What the numbers say

  • Fixed-size chunks crossed a section boundary 22.5% of the time at 200 characters and 81.3% at 2,000.
  • Heading-aware chunks never crossed a section, at any size.
  • On the lenient metric (any chunk from the right page counts), sizes above about 400 characters scored within a few points of each other.

All results

Fixed2008,17919822.5%0.5830.8100.6740.5830.810
Fixed4004,11939338.9%0.5030.7580.6020.7000.908
Fixed6002,76658550.9%0.4470.7000.5480.7300.931
Fixed8002,08677459.8%0.3910.6020.4750.7360.949
Fixed1,2001,4161,13970.8%0.2920.4480.3540.7360.941
Fixed1,6001,0831,48676.8%0.1970.3210.2470.7340.937
Fixed2,0008841,81881.3%0.1610.2380.1930.7450.935
Heading-aware2006,3672290.0%0.8770.9810.9210.5110.862
Heading-aware4003,9803670.0%0.8220.9770.8870.7440.918
Heading-aware6003,1494640.0%0.7990.9720.8720.7410.924
Heading-aware8002,7395330.0%0.7850.9640.8580.7340.922
Heading-aware1,2002,3986090.0%0.7630.9600.8460.7300.932
Heading-aware1,6002,2506490.0%0.7610.9560.8430.7360.936
Heading-aware2,0002,1766710.0%0.7570.9560.8400.7390.942

Setup

FastAPI English docs (152 Markdown files, 1.46M characters), 781 heading queries, BM25 (k1=1.5, b=0.75), Python 3.12, standard library only. No embeddings were tested. Script and raw output are in the Experiment files box.

Reproduce it

Read the full write-upFree lesson: ingestion and chunkingShare XLinkedIn

More benchmarks

All benchmarks →