Concept guide · RAG foundations
How do you choose a chunking strategy?
A slower explanation for when the short interview answer is not enough. Start with the intuition, then connect it to the technical idea and the way an interviewer expects you to reason about it.
ML area
Generative AI → Chunking strategy
Interview format
Core · Scenario
Question
GENAI-050
Start with the idea
In simple terms
Chunking decides which pieces of a document can be retrieved together. Small chunks are precise but may lose context; large chunks preserve context but bring more irrelevant text and cost.
Make it concrete
A policy exception should stay with the rule it modifies. Splitting every 500 characters can separate those two sentences even if the document’s heading makes the relationship obvious to a person.
Build the answer
How to approach it
- 1Inspect the document structure before choosing a splitter.
- 2Preserve the context and metadata a reader would need to interpret a chunk.
- 3Measure boundary misses, redundancy, retrieval rank, and answer quality.
Go one level deeper
The technical explanation
Start from document structure, headings, paragraphs, code blocks, list items, and table boundaries, then enforce token and size limits. Add parent titles, section paths, page numbers, and source identifiers to each child chunk. Compare fixed, recursive, semantic, and structure-aware strategies on retrieval recall and downstream answer quality; chunk size is not an aesthetic preference.
Say this in the interview
Start from document structure such as headings, paragraphs, code blocks, or table boundaries. Small chunks improve precision but can lose context; large chunks preserve context but add noise and tokens. Preserve parent titles and metadata, then compare strategies on labelled retrieval and answer cases.
Keep preparing