RecursiveCharacterTextSplitter vs MarkdownHeaderTextSplitter

The short answer

RecursiveCharacterTextSplitter is a size-aware general-purpose splitter. It tries larger textual boundaries first and moves to smaller separators when a piece is still too large. MarkdownHeaderTextSplitter is structure-aware: it groups Markdown content by selected headings and puts the heading path into each returned document's metadata.

For Markdown knowledge bases, a useful default is often to split on headings first, then apply recursive character splitting to sections that are too long. That preserves section context while still creating chunks small enough for your embedding and retrieval setup. Always inspect the actual output and evaluate retrieval quality against your own questions.

What each splitter returns

How RecursiveCharacterTextSplitter works

The recursive splitter tries a list of separators in order. Its usual default hierarchy favors paragraph breaks, then line breaks, spaces, and finally individual characters. It keeps dividing pieces that are longer than the configured size, using a finer boundary as needed, then merges smaller pieces while respecting the overlap setting where possible.

This strategy aims to avoid cutting sentences or paragraphs when a natural boundary is available. It is still a character-oriented heuristic by default, not a guarantee that every chunk has the same number of model tokens or represents one complete idea. If you need token-aware limits, configure a suitable length function or use a tokenizer-based splitter and verify chunks against the model that will consume them.

from langchain_text_splitters import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=900,
    chunk_overlap=150,
    add_start_index=True,
)

# documents is a list of langchain_core.documents.Document objects
chunks = splitter.split_documents(documents)
print(chunks[0].metadata)
print(chunks[0].page_content[:300])

The numbers above are an example starting point, not universal RAG settings. The default length measurement is characters. Chunk size, overlap, and embedding model limits should be tuned together using representative documents and test questions.

How MarkdownHeaderTextSplitter works

The header splitter scans Markdown lines for the heading markers you specify. As it moves through the document, it tracks the current heading hierarchy and attaches those heading values to the resulting Document metadata. Adjacent content under the same heading path is grouped together by default. You can choose which heading levels to track and whether the recognized headings remain in page_content with the strip_headers option.

This is different from MarkdownTextSplitter, which is a Markdown-flavored recursive character splitter. The class in this guide is specifically MarkdownHeaderTextSplitter: it uses heading hierarchy and returns section documents. It is not itself a size-limiting subclass of the regular TextSplitter interface.

from langchain_text_splitters import MarkdownHeaderTextSplitter

markdown = """# Retrieval
## Chunking
Split a source document into useful passages.
## Metadata
Keep source and section details with each passage.
"""

headers = [
    ("#", "Header 1"),
    ("##", "Header 2"),
]
header_splitter = MarkdownHeaderTextSplitter(
    headers_to_split_on=headers,
    strip_headers=True,
)

sections = header_splitter.split_text(markdown)
for section in sections:
    print(section.metadata)
    print(section.page_content)

With strip_headers=True, heading text is removed from section content and recorded in metadata. Metadata is valuable for filtering, citations, and display, but many vector stores embed page_content rather than metadata. If a retrieved passage needs to make sense alone, consider retaining headings in the content or adding a concise heading breadcrumb before embedding.

A practical two-stage pipeline for Markdown RAG

First preserve the document outline. Then split any oversized sections to a size target. Passing the section Documents into split_documents carries their metadata into the smaller chunks, so each passage can still be traced to its Markdown headings.

from langchain_text_splitters import (
    MarkdownHeaderTextSplitter,
    RecursiveCharacterTextSplitter,
)

headers = [("#", "Header 1"), ("##", "Header 2"), ("###", "Header 3")]
markdown_splitter = MarkdownHeaderTextSplitter(
    headers_to_split_on=headers,
    strip_headers=False,
)
sections = markdown_splitter.split_text(markdown_text)

size_splitter = RecursiveCharacterTextSplitter(
    chunk_size=900,
    chunk_overlap=120,
    add_start_index=True,
)
chunks = size_splitter.split_documents(sections)

for chunk in chunks[:3]:
    print(chunk.metadata)       # heading path plus splitter metadata
    print(chunk.page_content)   # bounded passage with heading context

Keeping headings with strip_headers=False makes this example easier to inspect and gives the chunk text some topic context. If you prefer clean body text, set it to True and build a heading breadcrumb from metadata before embedding. When you do that, avoid duplicating an entire long heading tree into every chunk; retain only the useful section path.

The recursive splitter measures and overlaps each section independently. Overlap therefore does not cross a Markdown heading boundary in this pipeline, which is often desirable because separate sections should not silently blend. If your content depends on context introduced in a parent section, include that parent context deliberately or use a parent-child retrieval design.

Which splitter should you choose?

Choose RecursiveCharacterTextSplitter when

• Your source is plain text, extracted PDF text, or content without reliable Markdown headings.
• You need a straightforward chunk-size and overlap control.
• You want a final size-control pass after a structure-aware parser.
• You need to preserve metadata from documents that already carry source IDs or other fields.

Choose MarkdownHeaderTextSplitter when

• Your source is Markdown with meaningful heading levels.
• Answers should retain the chapter, section, or subsection that introduced a fact.
• You want heading metadata for filtering, result labels, or source citations.
• Section-aware boundaries matter more than guaranteeing a maximum chunk size in a single step.

For long Markdown pages, combine them in that order: headings first, size limits second. For short, consistently sized sections, the header splitter may be enough on its own. For scanned documents, HTML, tables, code, or PDFs with complex layout, consider a parser or format-specific splitter before either of these.

Chunking mistakes to avoid

• Treating character counts as token counts: character length is only a proxy for model tokens; text, code, and languages tokenize differently.
• Assuming a heading split controls size: one section can contain a whole chapter, so apply a size-aware pass when needed.
• Dropping section context: if headings are only metadata and metadata is not embedded, retrieved body text may be hard to interpret by itself.
• Using excessive overlap: large overlap can repeat nearly identical passages, increase storage, and crowd retrieval results.
• Choosing settings without retrieval checks: inspect chunks and test whether the right passages are retrieved for real questions.

Compare chunking variants on a small evaluation set: record representative questions, the source section that should answer each one, and whether the retriever returns it near the top. Then measure answer quality and citation accuracy. A visually tidy chunk is not automatically a useful retrieval unit.

How this fits into a RAG system

Chunking sits between document parsing and indexing. After splitting, a pipeline typically adds source metadata, embeds the text, stores vectors and metadata, retrieves relevant passages for a query, and supplies those passages to a language model. Splitting can improve retrieval, but it cannot repair missing source content, weak embeddings, a poor retrieval query, or unsupported answers by itself.

For the bigger picture, read the RAG beginner guide, RAG pipeline walkthrough, and hybrid search guide. You can also learn why Markdown can be a useful format for AI workflows.

Official references

• LangChain reference: RecursiveCharacterTextSplitter
• LangChain reference: MarkdownHeaderTextSplitter
• LangChain reference: TextSplitter options and size measurement