Package sk.iway.iwcm.rag.indexing

Class MarkdownChunker

java.lang.Object

sk.iway.iwcm.rag.indexing.MarkdownChunker

@Component
public class MarkdownChunker
extends Object

Splits Markdown at block boundaries, preserving fitting structures and their heading context.

Constructor Summary

Constructors

Constructor	Description
MarkdownChunker(MarkdownContentExtractor extractor, SlidingWindowChunker fallback)

Method Summary

Modifier and Type	Method	Description
List<SlidingWindowChunker.Chunk>	chunkWithOffsets(String text)	Splits cleaned Markdown using the configured size and overlap.
List<SlidingWindowChunker.Chunk>	chunkWithOffsets(String text, int chunkSize, int overlap)	Groups fitting subsections and adjacent short level-two sections, preserving their structures.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

MarkdownChunker

public MarkdownChunker(MarkdownContentExtractor extractor,
 SlidingWindowChunker fallback)

Method Details

chunkWithOffsets

public List<SlidingWindowChunker.Chunk> chunkWithOffsets(String text)

Splits cleaned Markdown using the configured size and overlap.

Parameters:

text - cleaned Markdown

Returns:

chunks with source offsets compatible with heading-context extraction

chunkWithOffsets

public List<SlidingWindowChunker.Chunk> chunkWithOffsets(String text,
 int chunkSize,
 int overlap)

Groups fitting subsections and adjacent short level-two sections, preserving their structures. Level-one headings remain boundaries; overlap resets between unmerged sections. Blocks larger than the 50% size allowance use the ordinary bounded text splitter.

Parameters:

text - cleaned Markdown; code indentation is preserved

chunkSize - target body size; nonpositive values disable splitting

overlap - target overlap within a section; one whole trailing block may exceed it

Returns:

passages with offsets in the normalized, trimmed source, excluding heading prefixes