the retrieval-sized piece of a source document - the unit that gets embedded, indexed, retrieved, and cited; small enough to fit prompts, large enough to carry meaning.
Document to chunks to leaf entities - where retrieval meets accountability.
Etymology and backstory
chunking is the unglamorous craft at the bottom of every RAG system - split documents by tokens, sentences, headings, or semantic boundaries, with overlaps to survive boundary cuts. Chunk size trades recall against precision against prompt budget, and practitioners learn that most "bad RAG" is really bad chunking. The word itself is plain English that stuck because nothing fancier was needed.
Ecosystem
every RAG framework (LangChain, LlamaIndex) ships chunking strategies; evaluation suites measure retrieval quality per chunking choice; Eric's own troubleshooting purpose for signatures - which store was slow, where a chunking error crept in, which chunk fed which answer - is chunk-level observability.
In codexproof
the chunk is the leaf entity of the evidence DAG - the thing whose bytes get canonical-ized, hashed to a cid, and signed by the domain's proxy at capture. Eric's own summary sentence, confirmed against the mechanics in Q18 of the QA file: BLAKE3 hashes the chunks, Ed25519 signs that (via the domain- separated cid). When an answer cites its evidence, it cites chunk cids.
---