Matching a chunking method to document structure

Databricks states that there is no one-size-fits-all approach: the right method is iterative and depends on the use case and the data. The structure already in the document is the best first guide.

Some decision rules follow from that table:

• If the source has headings, use them. Format-specific chunking gets its boundaries free from the document, and section names become useful metadata.
• If the source is plain prose, start at paragraph level. It can help preserve coherence, as paragraphs often contain related information.
• If parsing has flattened the document, repair the parsing first. Poor parsing, such as missing tables or broken formatting, directly lowers retrieval quality, and no splitter can recover structure that is gone. The ai_parse_document function is the Databricks recommendation for PDFs and complex documents.
• Clean before splitting. Remove boilerplate such as headers, footers and page numbers, keep headings, lists and tables, and keep semantic boundaries intact.

A token-counted split is still useful as a safety net beneath a structural method, or for a first baseline. The Databricks example notebook uses this pattern with tiktoken, here with a smaller limit than the notebook’s 8192:

import tiktoken

max_chunk_tokens = 512
encoding = tiktoken.get_encoding("cl100k_base")

def chunk_text(text):
    tokens = encoding.encode(text)
    chunks = []
    while tokens:
        chunk_tokens = tokens[:max_chunk_tokens]
        chunks.append(encoding.decode(chunk_tokens))
        tokens = tokens[max_chunk_tokens:]
    return chunks

This loop has no overlap and no regard for sentences, so it is a guard rather than a finished strategy. It also needs a new unique chunk ID for each piece: the same notebook builds one from the source ID and a counter, as in f"{row['id']}_{chunk_no}".

Common trap: Reading “recursive” splitters as a form of semantic chunking - the Databricks page lists RecursiveCharacterTextSplitter under paragraph-based chunking, and its fixed-size example also uses that class with chunk_size=100 and chunk_overlap=20. The page groups it with LangChain’s character splitters, while SemanticChunker is its semantic example.

Get the whole book

This note is one section of Ultra Transcenders DBX-GENAI: Databricks Generative AI Engineer Associate, an independent study guide that explains every topic the exam covers by technology, with comparison tables, diagrams and the common traps, plus a glossary linked to the Databricks documentation.

About the book · DBX-GENAI terms in the glossary · All DBX-GENAI study notes