Skip to main content
Deepgram's Docs

Search documentation

Type to search this documentation.

On this pageOverview

Pipecat and Deepgram

This guide walks you through building a voice AI agent that uses Pipecat for pipeline orchestration and Deepgram for speech-to-text (STT) and text-to-speech (TTS). By the end, you have a working voice agent that listens to a user, generates a response with an LLM, and speaks back in real-time.

Pipecat is an open-source Python framework for building voice and multimodal AI agents. It connects STT, LLM, and TTS services into a real-time pipeline and handles audio transport, turn-taking, and interruption detection.

Before you begin

Section titled “Before you begin”

You need:

Install or update the Pipecat CLI:

Bash

To update the CLI use:

Bash

Choose your developer experience

Section titled “Choose your developer experience”

Creating a Pipecat + Deepgram integration can be accomplished by using several approaches. Choose the developer experience from the guides below that best fits your style. Note that all paths share the same prerequisite: the Pipecat CLI.

  1. Build with a Coding Agent
  2. Use the quickstart CLI command
  3. Scaffold a new Pipecat project with the CLI

Build with a Coding Agent

Section titled “Build with a Coding Agent”

You can use AI coding tools like Claude Code or Codex to generate your Pipecat agent code. Rather than relying on the tool’s training data, you give it live context from the Pipecat documentation.

  1. Follow the Pipecat getting started guide to set up AI tools, connect the Pipecat Context Hub, and initialize a project.
  2. Start a coding session with a prompt like the example below.
I'm building a phone assistant for my flower shop, Field & Flower, that
takes customer orders.
The bot should be able to:
  - list the available bouquets
  - check if a specific flower is in stock
  - add a flower to the order
  - get a summary of the order
  - set the delivery details
  - place the order
  - end the call
When the call starts, the bot greets the caller with exactly:
"This is Field & Flower, your local flower shop. How can I help you today?"
Services:
  - Twilio for phone calls
  - STT: Deepgram Flux
  - LLM: OpenAI
  - TTS: Deepgram Flux
  - Deploy to Pipecat Cloud
This is a demo: use a mock backend for the flower data, and "place the
order" only needs to log the order.

The init command creates a GETTING_STARTED.md file with additional guidance for your coding agent.

Use the quickstart CLI command

Section titled “Use the quickstart CLI command”

The quickstart uses Deepgram for STT but Cartesia for TTS. Follow the instruction from the Pipecat Quickstart documentation, then switch to Deepgram using the steps below.

To switch TTS to Deepgram, open bot.py and find the Cartesia TTS setup:

Python
# Remove this:
from pipecat.services.cartesia.tts import CartesiaTTSService
tts = CartesiaTTSService(
    api_key=os.getenv("CARTESIA_API_KEY"),
    settings=CartesiaTTSService.Settings(
        voice="71a7ad14-091c-4e8e-a314-022ece01c121",
    ),
)

Replace it with:

Python
from pipecat.services.deepgram.flux.tts import DeepgramFluxTTSService
tts = DeepgramFluxTTSService(
    api_key=os.getenv("DEEPGRAM_API_KEY"),
    settings=DeepgramFluxTTSService.Settings(
        voice=os.getenv("DEEPGRAM_VOICE_ID", "flux-alexis-en")
    ),
)

Remove CARTESIA_API_KEY from your .env file — it is no longer needed. No other changes are required: the STT service already uses Deepgram, and the rest of the pipeline stays the same.

Flux TTS voices use the model string format flux-{voice}-{language}, such as flux-alexis-en. This differs from the aura-2-{voice}-{language} format used by Deepgram’s Aura-2 voices. Browse the Flux TTS voice catalog to choose a different voice.

Continue building by adding a Pipecat Client

Use Flux for turn detection

Section titled “Use Flux for turn detection”

Flux is Deepgram’s conversational STT model with built-in turn detection. It uses acoustic and semantic cues to determine when a speaker has finished their turn, resulting in more natural conversations.

To use Flux, replace DeepgramSTTService with DeepgramFluxSTTService in your bot.py:

Python
import os
from pipecat.services.deepgram.flux.stt import DeepgramFluxSTTService
stt = DeepgramFluxSTTService(
    api_key=os.getenv("DEEPGRAM_API_KEY"),
    settings=DeepgramFluxSTTService.Settings(
        min_confidence=0.3,
    ),
)

Scaffold a new Pipecat project with the CLI

Section titled “Scaffold a new Pipecat project with the CLI”

Step 1: Create the project

Section titled “Step 1: Create the project”

Scaffold a new project using the Pipecat CLI.

Bash

Step 2: Install dependencies

Section titled “Step 2: Install dependencies”

Navigate to the server directory inside your new project, create a virtual environment, and install the dependencies:

Bash

Step 3: Configure your environment

Section titled “Step 3: Configure your environment”

Copy the example environment file and fill in your API keys:

Bash

.env.example does not include a voice setting, so add one — the scaffolded bot reads DEEPGRAM_VOICE_ID and will not synthesize speech without it:

Bash

Replace the placeholder values with your API keys:

The remaining values are defaults you can change later.

Step 4: Run the agent

Section titled “Step 4: Run the agent”

Start the bot from the server directory:

Bash

Step 5: Test the conversation

Section titled “Step 5: Test the conversation”

Open the local URL printed in your terminal, then:

  1. Select Daily from the Transport list and click Connect.
  2. Allow microphone access and speak to your agent.
  3. Ask a question and confirm the agent responds with speech.
  4. Speak while the agent is talking — it should stop and listen.
  5. Pause after speaking — the agent should detect the end of your turn and respond.

Continue building by adding a Pipecat Client

Next Steps

Section titled “Next Steps”

Continue building with an agent.

Follow the Pipecat getting started guide and ready the Pipecat Context Hub.

Prompt your agent to add a Pipecat client framework.

Example prompt:

Add a pipecat client for React

Go further with Deepgram

Section titled “Go further with Deepgram”
  • Voices — Update DEEPGRAM_VOICE_ID in your .env file. Browse the Flux TTS voice catalog for flux-{voice}-{language} voices. If you switched to DeepgramTTSService for a non-English language, pick an Aura-2 voice instead (aura-2-{voice}-{language}).
  • Keyterm prompting — Improve recognition of domain-specific vocabulary by passing keyterms to Nova-3 via the STT service settings.
  • Speaker diarization — Assign a speaker identifier to each word in the transcript using diarization via the STT service settings.
  • Dynamic STT settings — Pipecat supports updating Deepgram STT settings without reconnecting. See the Pipecat Deepgram STT guide for details.

Resources

Section titled “Resources”
Suggest an edit

Propose a replacement for this page. The site team reviews it before applying any changes.

Export
Documentation menu