Validate a Deepgram SageMaker Endpoint

Once your SageMaker endpoint reaches InService, run a client script against it to confirm that inference works end to end. Deepgram publishes a ready-made set of test clients in the dg-sagemaker repository. Each client wraps the SageMaker bidirectional streaming or batch (HTTP) invocation API with the payload shape that the target Deepgram model expects, so you can focus on verifying the endpoint rather than writing transport code.

The repository is organized by product and by language. Pick the script that matches the model you deployed (Flux, Nova-3, or Aura) and the language you want to work in (Python, Node.js, or Java). The scripts are load-testing clients first and functional smoke tests second — running any of them with a single connection is the fastest way to prove that your endpoint accepts traffic and returns results.

• A Deepgram SageMaker endpoint in InService status. See Deploy Deepgram on Amazon SageMaker.

• AWS IAM permissions to invoke the endpoint and, optionally, to list endpoints in the target region:

  • sagemaker:InvokeEndpoint
  • sagemaker:InvokeEndpointWithResponseStream
  • sagemaker:InvokeEndpointWithBidirectionalStream
  • sagemaker:ListEndpoints and sagemaker:DescribeEndpoint (used by list-endpoints helpers)

• git available on your local machine.

Each Deepgram product has a dedicated script because the SageMaker payload shape and the protocol on top of it differ per model. Run the script that matches the model you deployed.

Install only the tools you need for the script you plan to run.

cd python-stt   # or python-flux, python-tts
uv sync

The scripts use the standard AWS credential chain — environment variables, shared credentials file, or an attached IAM role. Configure credentials with whichever mechanism you prefer:

aws configure

Or export them for the current shell:

export AWS_ACCESS_KEY_ID=...
export AWS_SECRET_ACCESS_KEY=...
export AWS_SESSION_TOKEN=...   # if using temporary credentials

Verify the identity the scripts will use:

aws sts get-caller-identity

Every script accepts a --region flag. Defaults differ by script: flux_stress.py and stt_microphone_stress.py default to us-east-1; stt_wav_stress.py and tts_stress.py default to us-east-2; invoke_test.py has no default and requires --region (or AWS_REGION). Make sure the region you pass matches the region where your SageMaker endpoint was deployed — an endpoint name is only resolvable within the region it was created in.

If you are not sure which endpoints exist in a region, use the Flux client’s helper subcommand to list them:

cd python-flux
uv run flux_stress.py list-endpoints --region us-east-1

Once tooling and credentials are in place, invoke the script for your product. The example below shows the simplest single-connection invocation for the Flux Python client. The Nova-3 and Aura clients follow a similar pattern but with different flags, defaults, and input formats — refer to the dg-sagemaker repository and each subdirectory’s README.md for the full command reference.

cd python-flux

# Stream a WAV file at real-time pace
uv run flux_stress.py file your-endpoint-name \
  --file audio.wav \
  --region us-east-1

# Or stream live microphone audio
uv run flux_stress.py microphone your-endpoint-name --region us-east-1

For best compatibility, use a 16-bit PCM WAV file. Convert other formats with:

ffmpeg -i input.mp3 -ar 16000 -ac 1 -sample_fmt s16 audio.wav

A successful test produces:

• Speech-to-text (Flux, Nova-3): transcript payloads streamed back to the client. Final transcripts and interim results (or Flux TurnInfo events) are logged to the console.
• Text-to-speech (Aura): audible speech playback from the selected connection and a steady stream of synthesized audio chunks logged to the console.

If the script hangs, errors on connection, or fails to authenticate, see Troubleshooting.

• ResourceNotFound or endpoint lookup failures — confirm the endpoint name and that your shell is targeting the correct region. Run aws sagemaker describe-endpoint --endpoint-name your-endpoint-name --region your-region to verify.
• AccessDeniedException — confirm your IAM identity has sagemaker:InvokeEndpoint (and the bidirectional/streaming variants) for the endpoint’s ARN.
• Endpoint not yet in service — wait until the endpoint status is InService. Deployment typically takes several minutes.
• Audio format errors — try changing the input audio format to 16-bit PCM WAV, instead of other compressed formats. Use the ffmpeg command as shown above.
• Server-side errors — check the Amazon CloudWatch Logs for the SageMaker endpoint to diagnose container-level issues.