Chunking

This module provides a set of classes for chunking a long text into smaller chunks.

The Chunker Interface

ChunkerBase is the base class for all chunkers. It provides a simple interface for chunking a text into smaller chunks. The chunking process is controlled by a configuration object that is passed to the chunker's constructor.

Chunkers

Sentence Splitters

This submodule provides a set of useful tools for splitting a text into sentences.

General Configuration

The configuration provides a general interface for loading and configurate the chunker or the sentence splitter.