Document chunking for RAG: preserve context and source citations

Preserve enough context to answer the question

A chunk is the portion of a document a retrieval system selects as one unit. Splitting makes it possible to select relevant material, but the boundary matters. Cut between a rule and its exception, or a number and its label, and the retrieved text can mislead.

Start with the evidence needed for a question. For "what is the cancellation period?", the number alone is inadequate if the next paragraph limits it to a particular service tier.

Inspect boundaries that carry meaning

Use these questions to choose boundaries for your documents. They do not prescribe one way to split every file. LangChain's splitter documentation describes length-based and document-structure approaches. Test a strategy against your own material before tuning it.

Use a boundary test with a visible exception

Create a short synthetic policy: standard support covers weekdays. A second sentence excludes public holidays. Ask whether support is available on a holiday. If retrieval only supplies the first sentence, the agent cannot properly interpret the rule.

Compare a paragraph-preserving split with a smaller fixed split. Keep the model, question and source revision constant. Save the retrieved text, not just the final answer. This is an evaluation exercise, not a result for either strategy.

Carry source identity into each retrieved unit

Each unit should lead back to an original document and a stable location or passage. Preserve section headings where they establish scope. Record the document revision or capture date where changes matter.

Overlap can keep adjacent context together, but it also creates repeated text. Avoid interpreting three overlapping passages as three independent sources. Deduplicate evidence when writing a final answer and keep the original source as the authority.

Adjust chunk size using failed queries

If relevant passages are absent, examine import coverage and retrieval before concluding that every chunk is too large. If the answer mixes unrelated sections, a narrower unit or better source metadata may help. If qualifiers disappear, preserve the surrounding structure.

There is no single word count that makes every document searchable. Build a test set with prose, tables, short notes and changed facts. Compare relevant retrieval, supported answers and context sent to the agent. Compare each change with a simple split, and keep it only if the results justify the extra work.

Evaluate your import workflow

The retrieval evaluation kit shows how to keep questions, relevance decisions and results separate. Its arithmetic example is synthetic and does not benchmark Gigabrain.

For Gigabrain, use the supported import/index workflow rather than editing internal retrieval files to match a generic tutorial. Inspect the product mechanism and setup guide. Compare plans after verifying that the system fits your sources.

Compare two splits of one source

Consider this fictional sentence: "Weekday support covers the inspection team, except on public holidays." A split after "team" separates the offer from its exception.

This is a constructed boundary example, not a measured retrieval improvement. Both text variants come from the same supplied sentence.

The boundary worksheet includes the source, two candidate splits and questions for a repeatable review. Test actual retrieval separately before choosing a chunking configuration.

Retain the location as well as the words

The connected-source experiment returns a document identity, quotation and location. Those fields let a reviewer inspect the underlying source instead of relying on the chunk alone.

See the citation walkthrough for the changed-source failure. A convenient chunk boundary does not remove the need for source verification.