Text Chunking for TTS REST Optimization

When dealing with lengthy text inputs, latency can become an issue as the processing time increases with the length of the text. To address this challenge, one effective strategy is text chunking. Text chunking is the process of breaking down text inputs into smaller, manageable chunks before processing.

Code Examples

The following three code examples build upon each previous example in complexity to present strategies for using text chunking to optimize your Text-to-Speech applications.

Chunk By Maximum Number of Characters

This example breaks down lengthy text inputs into chunks determined by a maximum number of characters.

This is a straightforward example which does not take into consideration characteristics of the text structure, such as clause and sentence boundaries. For some types of text, this is acceptable and will not have a negative effect on the quality of speech.

Python

# For more Python SDK migration guides, visit:
# https://github.com/deepgram/deepgram-python-sdk/tree/main/docs
from deepgram import DeepgramClient
# Define the maximum chunk size (in characters) for text chunking
MAX_CHUNK_SIZE = 200
input_text = "Our story begins in a peaceful woodland kingdom where a lively squirrel named Frolic made his abode high up within a cedar tree's embrace. He was not a usual woodland creature, for he was blessed with an insatiable curiosity and a heart for adventure. Nearby, a glistening river snaked through the landscape, home to a wonder named Splash - a silver-scaled flying fish whose ability to break free from his water-haven intrigued the woodland onlookers. This magical world moved on a rhythm of its own until an unforeseen circumstance brought Frolic and Splash together. One radiant morning, while Frolic was on his regular excursion, and Splash was making his aerial tours, an unpredictable wave playfully tossed and misplaced Splash onto the riverbank. Despite his initial astonishment, Frolic hurriedly and kindly assisted his new friend back to his watery abode. Touched by Frolic's compassion, Splash expressed his gratitude by inviting his friend to share his world. As Splash perched on Frolic's back, he tasted of the forest's bounty, felt the sun’s rays filter through the colors of the trees, experienced the conversations amidst the woods, and while at it, taught the woodland how to blur the lines between earth and water."
def chunk_text(text, chunk_size):
    chunks = []
    words = text.split()
    current_chunk = ''
    for word in words:
        if len(current_chunk) + len(word) <= chunk_size:
            current_chunk += ' ' + word
        else:
            chunks.append(current_chunk.strip())
            current_chunk = word
    if current_chunk:
        chunks.append(current_chunk.strip())
    return chunks
def main():
    try:
        # Create a Deepgram client using the API key
        deepgram = DeepgramClient(api_key="DEEPGRAM_API_KEY")
        # Chunk the text into smaller parts
        text_chunks = chunk_text(input_text, MAX_CHUNK_SIZE)
        # Synthesize audio for each chunk
        for i, chunk in enumerate(text_chunks):
            print(f"\nProcessing chunk {i + 1}...{chunk}\n")
            filename = f"chunk_{i + 1}.mp3"
            response = deepgram.speak.v1.audio.generate(
                text=chunk,
                model="aura-2-thalia-en"
            )
            # Save the audio file
            with open(filename, "wb") as audio_file:
                for audio_chunk in response:
                    audio_file.write(audio_chunk)
            print(f"Audio saved to {filename}")
    except Exception as e:
        print(f"Exception: {e}")
if __name__ == "__main__":
    main()

Chunk By Clauses and Sentence Boundaries

In this example, the aim is to preserve naturalness of speech by chunking the text based on clause and sentence boundaries. When people speak, they tend to pause at the end of a clause or a sentence, so this strategy is helpful when working with texts that contain complex sentences in a narrative style.

The regular expression in the code tells the program to break the text into chunks based on the following:

The punctuation marks of period ., question mark ?, explanation mark !, or semicolon ;
A comma , + single whitespace + coordinating conjunctions and, but, or, nor, for, yet, so

These are two grammatical rules for identifying clauses, but you may decide to include more.

Python

# For more Python SDK migration guides, visit:
# https://github.com/deepgram/deepgram-python-sdk/tree/main/docs
import re
from deepgram import DeepgramClient
input_text = "Our story begins in a peaceful woodland kingdom where a lively squirrel named Frolic made his abode high up within a cedar tree's embrace. He was not a usual woodland creature, for he was blessed with an insatiable curiosity and a heart for adventure. Nearby, a glistening river snaked through the landscape, home to a wonder named Splash - a silver-scaled flying fish whose ability to break free from his water-haven intrigued the woodland onlookers. This magical world moved on a rhythm of its own until an unforeseen circumstance brought Frolic and Splash together. One radiant morning, while Frolic was on his regular excursion, and Splash was making his aerial tours, an unpredictable wave playfully tossed and misplaced Splash onto the riverbank. Despite his initial astonishment, Frolic hurriedly and kindly assisted his new friend back to his watery abode. Touched by Frolic's compassion, Splash expressed his gratitude by inviting his friend to share his world. As Splash perched on Frolic's back, he tasted of the forest's bounty, felt the sun’s rays filter through the colors of the trees, experienced the conversations amidst the woods, and while at it, taught the woodland how to blur the lines between earth and water."
CLAUSE_BOUNDARIES = r'\.|\?|!|;|, (and|but|or|nor|for|yet|so)'
def chunk_text_by_clause(text):
    # Find clause boundaries using regular expression
    clause_boundaries = re.finditer(CLAUSE_BOUNDARIES, text)
    boundaries_indices = [boundary.start() for boundary in clause_boundaries]
    chunks = []
    start = 0
    for boundary_index in boundaries_indices:
        chunks.append(text[start:boundary_index + 1].strip())
        start = boundary_index + 1
    # Append the remaining part of the text
    chunks.append(text[start:].strip())
    return chunks
def main():
    try:
        # Create a Deepgram client using the API key
        deepgram = DeepgramClient(api_key="DEEPGRAM_API_KEY")
        # Chunk the text into smaller parts
        text_chunks = chunk_text_by_clause(input_text)
        # Synthesize audio for each chunk
        for i, chunk in enumerate(text_chunks):
            print(f"\nProcessing chunk {i + 1}...{chunk}\n")
            filename = f"chunk_{i + 1}.mp3"
            response = deepgram.speak.v1.audio.generate(
                text=chunk,
                model="aura-2-thalia-en"
            )
            # Save the audio file
            with open(filename, "wb") as audio_file:
                for audio_chunk in response:
                    audio_file.write(audio_chunk)
            print(f"Audio saved to {filename}")
    except Exception as e:
        print(f"Exception: {e}")
if __name__ == "__main__":
    main()

Dynamic Chunking

The goal of dynamic chunking is to adjust the chunk sizes dynamically based on various factors, which may include adaptive rules or algorithms to determine how to split the text into chunks.

This next example implements a more flexible chunking strategy that adjusts chunk sizes dynamically based on the length and structure of the input text. It retains the rule from the previous example - to chunk based on clause/sentence boundaries - but it then looks at each chunk and determines the character count of the chunk. If the count exceeds a maximum character length, it chunks further into subchunks, where subchunks are defined by a comma but cannot be less than three characters.

Python

# For more Python SDK migration guides, visit:
# https://github.com/deepgram/deepgram-python-sdk/tree/main/docs
import re
from deepgram import DeepgramClient
input_text = "Our story begins in a peaceful woodland kingdom where a lively squirrel named Frolic made his abode high up within a cedar tree's embrace. He was not a usual woodland creature, for he was blessed with an insatiable curiosity and a heart for adventure. Nearby, a glistening river snaked through the landscape, home to a wonder named Splash - a silver-scaled flying fish whose ability to break free from his water-haven intrigued the woodland onlookers. This magical world moved on a rhythm of its own until an unforeseen circumstance brought Frolic and Splash together. One radiant morning, while Frolic was on his regular excursion, and Splash was making his aerial tours, an unpredictable wave playfully tossed and misplaced Splash onto the riverbank. Despite his initial astonishment, Frolic hurriedly and kindly assisted his new friend back to his watery abode. Touched by Frolic's compassion, Splash expressed his gratitude by inviting his friend to share his world. As Splash perched on Frolic's back, he tasted of the forest's bounty, felt the sun’s rays filter through the colors of the trees, experienced the conversations amidst the woods, and while at it, taught the woodland how to blur the lines between earth and water."
CLAUSE_BOUNDARIES = r'\.|\?|!|;|, (and|but|or|nor|for|yet|so)'
MAX_CHUNK_LENGTH = 100
def chunk_text_dynamically(text):
    # Find clause boundaries using regular expression
    clause_boundaries = re.finditer(CLAUSE_BOUNDARIES, text)
    boundaries_indices = [boundary.start() for boundary in clause_boundaries]
    chunks = []
    start = 0
    # Add chunks until the last clause boundary
    for boundary_index in boundaries_indices:
        chunk = text[start:boundary_index + 1].strip()
        if len(chunk) <= MAX_CHUNK_LENGTH:
            chunks.append(chunk)
        else:
            # Split by comma if it doesn't create subchunks less than three words
            subchunks = chunk.split(',')
            temp_chunk = ''
            for subchunk in subchunks:
                if len(temp_chunk) + len(subchunk) <= MAX_CHUNK_LENGTH:
                    temp_chunk += subchunk + ','
                else:
                    if len(temp_chunk.split()) >= 3:
                        chunks.append(temp_chunk.strip())
                    temp_chunk = subchunk + ','
            if temp_chunk:
                if len(temp_chunk.split()) >= 3:
                    chunks.append(temp_chunk.strip())
        start = boundary_index + 1
    # Split remaining text into subchunks if needed
    remaining_text = text[start:].strip()
    if remaining_text:
        remaining_subchunks = [remaining_text[i:i+MAX_CHUNK_LENGTH] for i in range(0, len(remaining_text), MAX_CHUNK_LENGTH)]
        chunks.extend(remaining_subchunks)
    return chunks
def main():
    try:
        # Create a Deepgram client using the API key
        deepgram = DeepgramClient(api_key="DEEPGRAM_API_KEY")
        # Chunk the text into smaller parts
        text_chunks = chunk_text_dynamically(input_text)
        # Synthesize audio for each chunk
        for i, chunk in enumerate(text_chunks):
            print(f"\nProcessing chunk {i + 1}...{chunk}\n")
            filename = f"chunk_{i + 1}.mp3"
            response = deepgram.speak.v1.audio.generate(
                text=chunk,
                model="aura-2-thalia-en"
            )
            # Save the audio file
            with open(filename, "wb") as audio_file:
                for audio_chunk in response:
                    audio_file.write(audio_chunk)
    except Exception as e:
        print(f"Exception: {e}")
if __name__ == "__main__":
    main()

Chunking + Sequential Playback

Instead of saving each chunk as an audio file, you can play each chunk in sequence. This example collects each complete MP3 response before decoding and playing it.

The text is chunked by sentence boundaries, then each chunk is sent to Deepgram and played in order. To begin playback when the first byte arrives, use Real-Time TTS with WebSockets.

Python

# For more Python SDK migration guides, visit:
# https://github.com/deepgram/deepgram-python-sdk/tree/main/docs
import io
import re
from deepgram import DeepgramClient
from pydub import AudioSegment
from pydub.playback import play
input_text = "Our story begins in a peaceful woodland kingdom where a lively squirrel named Frolic made his abode high up within a cedar tree's embrace. He was not a usual woodland creature, for he was blessed with an insatiable curiosity and a heart for adventure. Nearby, a glistening river snaked through the landscape, home to a wonder named Splash - a silver-scaled flying fish whose ability to break free from his water-haven intrigued the woodland onlookers. This magical world moved on a rhythm of its own until an unforeseen circumstance brought Frolic and Splash together. One radiant morning, while Frolic was on his regular excursion, and Splash was making his aerial tours, an unpredictable wave playfully tossed and misplaced Splash onto the riverbank. Despite his initial astonishment, Frolic hurriedly and kindly assisted his new friend back to his watery abode. Touched by Frolic's compassion, Splash expressed his gratitude by inviting his friend to share his world. As Splash perched on Frolic's back, he tasted of the forest's bounty, felt the sun’s rays filter through the colors of the trees, experienced the conversations amidst the woods, and while at it, taught the woodland how to blur the lines between earth and water."
def chunk_text_by_sentence(text):
    # Find sentence boundaries using regular expression
    sentence_boundaries = re.finditer(r'(?<=[.!?])\s+', text)
    boundaries_indices = [boundary.start() for boundary in sentence_boundaries]
    chunks = []
    start = 0
    # Split the text into chunks based on sentence boundaries
    for boundary_index in boundaries_indices:
        chunks.append(text[start:boundary_index + 1].strip())
        start = boundary_index + 1
    chunks.append(text[start:].strip())
    return chunks
def synthesize_audio(text):
    # Create a Deepgram client using the API key
    deepgram = DeepgramClient(api_key="DEEPGRAM_API_KEY")
    # Synthesize audio and stream the response
    response = deepgram.speak.v1.audio.generate(
        text=text,
        model="aura-2-thalia-en"
    )
    # Collect the iterator because pydub decodes a complete MP3 file
    audio_buffer = io.BytesIO(b"".join(response))
    # Load audio from buffer using pydub
    audio = AudioSegment.from_mp3(audio_buffer)
    return audio
def main():
    # Chunk the text into smaller parts
    chunks = chunk_text_by_sentence(input_text)
    # Synthesize each chunk into audio and play the audio
    for chunk_text in chunks:
        audio = synthesize_audio(chunk_text)
        play(audio)
if __name__ == "__main__":
    main()

Considerations

When using text chunking as a strategy to minimize latency, some factors to keep in mind are the following:

Preserving naturalness of speech - Maintain proper pronunciation, intonation, and rhythm to enhance the user experience.
Contextual understanding - Analyze the structure and meaning of the text to identify natural breakpoints, such as sentence or clause boundaries, for dividing the text.
Dynamic chunking - Implement a flexible chunking strategy that adjusts chunk sizes dynamically based on the length and structure of the input text.
User expectations - Consider the preferences and needs of users, such as their tolerance for latency, the desired quality of synthesized speech, and their overall satisfaction with the application’s performance.

Search documentation

Flux TTS

Text Chunking for TTS REST Optimization

Code Examples

Chunk By Maximum Number of Characters

Chunk By Clauses and Sentence Boundaries

Dynamic Chunking

Chunking + Sequential Playback

Considerations

Flux TTS