Migrating From Amazon Web Services (AWS) Transcribe to Deepgram

This guide covers migrating your transcription processes from Amazon Web Services (AWS) Transcribe to Deepgram, including key differences, code examples, and best practices.

During the migration process, you will need to perform the following tasks:

Once you’ve selected your model, Deepgram provides many features and capabilities to help you transcribe and classify your audio. However, some capabilities and concepts are implemented differently from Amazon Transcribe.

Once you’ve selected your model, Deepgram provides many features and capabilities to help you transcribe and classify your audio. However, some capabilities and concepts are implemented differently from Amazon.

Both Deepgram and Amazon provide you with the following values by default:

• Words
• Timing
• Confidence

Additionally, Amazon provides the following values by default:

• Type

Interim Result

{
   "channel":{
      "alternatives":[
         {
            "confidence":0.99121094,
            "transcript":"section one of az",
            "words":[
               {
                  "confidence":0.9980469,
                  "end":1.2935,
                  "start":0.81589997,
                  "word":"section"
               },
               {
                  "confidence":0.9897461,
                  "end":1.6517,
                  "start":1.2935,
                  "word":"one"
               },
               {
                  "confidence":0.99121094,
                  "end":1.8507,
                  "start":1.6517,
                  "word":"of"
               },
               {
                  "confidence":0.5073242,
                  "end":1.9999375,
                  "start":1.8507,
                  "word":"az"
               }
            ]
         }
      ]
   },
   "channel_index":[
      0,
      1
   ],
   "duration":1.9999375,
   "is_final":false,
   "metadata":{
      "model_uuid":"4899aa60-f723-4517-9815-2042acc12a82",
      "request_id":"fbbe3da5-4a23-4fd3-a4eb-95f2fe8672f5"
   },
   "speech_final":false,
   "start":0.0
}

Final Result

{
   "channel":{
      "alternatives":[
         {
            "confidence":0.9941406,
            "transcript":"section one of az fable a new translation",
            "words":[
               {
                  "confidence":0.9975586,
                  "end":1.2970455,
                  "start":0.81813633,
                  "word":"section"
               },
               {
                  MORE WORDS...
               },
               {
                  "confidence":0.99853516,
                  "end":4.2314997,
                  "start":3.7315,
                  "word":"translation"
               }
            ]
         }
      ]
   },
   "channel_index":[
      0,
      1
   ],
   "duration":4.4,
   "is_final":true,
   "metadata":{
      "model_uuid":"4899aa60-f723-4517-9815-2042acc12a82",
      "request_id":"fbbe3da5-4a23-4fd3-a4eb-95f2fe8672f5"
   },
   "speech_final":false,
   "start":0.0
}

Deepgram and Amazon provide speech recognition models that are pre-trained or tuned to identify the words and phrases unique to a specific use case or domain. Deepgram creates our speech recognition models through transfer learning from our highly-performant general models. It is important to test multiple models to see which one meets the accuracy, performance, and scalability needs for your use case.

Deepgram provides:

• General
• Phone calls
• Meetings
• Voicemail
• Conversational AI
• Finance
• Video
• Whisper Cloud
• Custom Models

Amazon provides:

• Standard
• Medical
• Call Analytics

Deepgram provides acoustic-based search that identifies phrases by audio patterns rather than text matches. Unlike Amazon’s Call Analytics text-based search, this approach finds phrases even when transcription is imperfect, offering higher accuracy.

Deepgram’s Search feature uses a query that allows you to pass in a word or phrase to find and then returns results in the response JSON object. In the JSON response, we return information including the start time and end time of when that phrase was possibly uttered and a confidence rating for each match. So, in the response, you will see the query (the word or phrase you searched for), and then hits (an array of objects that give you the confidence, start, end, and snippet of each possible match to your search). You can include up to 25 search terms per request.

You can search for multiple terms individually: search=epistemology&search=warwick

You can search for a phrase. URL-encode the phrase when submitting it: search=social%20epistemology

As an example, we used a WAV audio file that contains the first 20 seconds of a college lecture on epistemology. The term “epistemology” in this audio file is sufficiently technical that our model will not transcribe it accurately, but our phonetic search will still be able to find it.

Input

search=epistemology

Response

"search":[
  {
    "query":"epistemology",
    "hits":[
      {"confidence":0.9348958,"start":10.725,"end":11.485,"snippet":"is a"},
      {"confidence":0.9348958,"start":13.965,"end":14.764999,"snippet":"epi"},
      {"confidence":0.9204282,"start":4.0074897,"end":4.805,"snippet":"social epi"},
      {"confidence":0.27662036,"start":8.792552,"end":10.365,"snippet":"us today is"},
      {"confidence":0.1319444,"start":17.205,"end":18.115,"snippet":"nature of knowledge"},
      {"confidence":0.0885417,"start":15.285,"end":16.085,"snippet":"branch philosophy"},
      {"confidence":0.045138836,"start":5.1240044,"end":5.722137,"snippet":"university of"},
      {"confidence":0.045138836,"start":5.6025105,"end":7.4367843,"snippet":"warwick and"},
      {"confidence":0.0,"start":1.0168257,"end":1.9339627,"snippet":"hello this is steve"},
      {"confidence":0.0,"start":7.277282,"end":8.27417,"snippet":"and the question"}
    ]
  }
]

Amazon Transcribe customers need to upgrade to Post Call Analytics (PCA) to have access to search capabilities. PCA currently supports the following search features:

• Search on call attributes such as time range, sentiment, or entities
• Search transcriptions

Both Deepgram and Amazon provide users the ability to improve the accuracy of specific keywords or vocabulary. Deepgram has two methods to perform this: our Keyword Boosting feature and model training. Amazon also has two methods for improving accuracy through custom vocabulary and creating a custom language model using their tools.

Deepgram’s Keywords feature improves transcription accuracy by focusing on specific terms you provide with your audio request.

Key points to consider:

• You can send up to 200 keywords per API request
• When listing keywords, you do not need to provide any “sounds like” spelling hints.
• Pro tip: If you want a keyword capitalized in your transcript, capitalize it in the keywords parameter.

For more than 200 unique keywords or domain-specific terms that Keyword Boosting doesn’t handle effectively, upgrade to a trained AI model. Deepgram can train a custom model using your audio samples and transcripts to learn your specific language, accents, and terminology.

Key points to consider:

• Deepgram trains models based on human-transcribed transcripts and the associated audio to achieve maximum accuracy.
• If you have only audio without transcribed text, we have an expert human transcription team that can transcribe the audio to create an excellent model training dataset.

Developers can use Amazon Transcribe custom vocabularies to improve transcription accuracy for one or more specific words. These are generally domain-specific terms, such as brand names and acronyms, proper nouns, and words that Amazon Transcribe isn’t rendering correctly.

Key points to consider:

• You can have up to 100 vocabularies per AWS account.

• You can create custom vocabularies in table or list format:

  • Table format: You must first upload your custom vocabulary file into an Amazon S3 bucket, then include the Amazon S3 URI of your vocabulary file in your request.
  • List format: If using the AWS Management Console, you must save your custom vocabulary file locally, then include the path to your custom vocabulary file in your request; if using the command line (AWS CLI or AWS SDK), you must include your list as comma-separated words within your request.

Amazon provides the option to use your own custom language model. These custom language models learn the context associated with a given word to more accurately transcribe that word. For customers with access to the data and with training and tuning programming expertise, this could be a viable option.

Key considerations:

• You must have separate sets of data on hand—one for training and one for tuning.

  • You can supply Amazon Transcribe with up to 2 GB of text data to train your model; this is referred to as training data.
  • Optionally, when you have no (or few) in-domain transcripts, you can provide Amazon Transcribe with up to 200 MB of text data to tune your model; this is referred to as tuning data.

• Your data must meet Amazon language, encoding, special character, and file size requirements, or your model training will fail.

  • Data must be in the same language as the model you want to create. For example, if you want to create a custom language model that transcribes audio in US English (en-US), all your text data must be in US English.
  • Data must be in plain text format with UTF-8 encoding.
  • Data must not contain any special characters or formatting, such as HTML tags.
  • Data must be a maximum combined total of 2 GB in size for training data and 200 MB for tuning data.

Both Deepgram and Amazon will provide you with the following values:

• transcript
• start_time (duration)
• end_time (duration)
• word (string)
• confidence (float)

In addition, by default, Amazon will provide:

• Type (pronunciation or punctuation)

Both Deepgram and Amazon provide pre-recorded, or batch, transcriptions. In addition to the limit differences for audio length and processing scale (listed below), Deepgram does not require you to store your audio in an Amazon S3 bucket. You can transcribe audio files from your preferred storage location—your laptop, VPC, or third-party cloud.

Amazon maintains the following limits for batch processing:

• Maximum audio length of 4 hours
• Maximum audio file size of 500 MB (call analytics)-2 GB
• Number of concurrent batch transcriptions of 100-250, depending on model

Deepgram maintains the following limits for batch processing:

• No limits to audio length
• Maximum audio file size of 2 GB per file
• Can scale to thousands of concurrent batch transcriptions
• Maximum processing time of 10 minutes per file (20 minutes for Whisper)

Both Deepgram and Amazon will provide you with the following values:

• transcript
• start_time (duration)
• end_time (duration)
• word (string)
• confidence (float)

Deepgram supports both transcription of files on your local machine and transcription of files stored remotely.

Deepgram

# For more Python SDK migration guides, visit:
# https://github.com/deepgram/deepgram-python-sdk/tree/main/docs
import os
from deepgram import DeepgramClient
import asyncio, json
DEEPGRAM_API_KEY = os.getenv('DEEPGRAM_API_KEY', 'YOUR_API_KEY')
PATH_TO_FILE = 'some/file.wav'
async def main():
    # Initializes the Deepgram SDK
    deepgram = DeepgramClient(api_key=DEEPGRAM_API_KEY)
    # Open the audio file
    with open(PATH_TO_FILE, 'rb') as audio:
        # ...or replace mimetype as appropriate
        payload = {'buffer': audio}
        response = await deepgram.listen.prerecorded.v('1').transcribe_file(
            payload,
            model='nova-2',
            punctuate=True
        )
        print(json.dumps(response, indent=4))
asyncio.run(main())

To see more, visit Deepgram’s Python SDK GitHub repo.

Amazon

import time
import boto3
def transcribe_file(job_name, file_uri, transcribe_client):
    transcribe_client.start_transcription_job(
        TranscriptionJobName = job_name,
        Media = {
            'MediaFileUri': file_uri
        },
        MediaFormat = 'flac',
        LanguageCode = 'en-US'
    )
    max_tries = 60
    while max_tries > 0:
        max_tries -= 1
        job = transcribe_client.get_transcription_job(TranscriptionJobName = job_name)
        job_status = job['TranscriptionJob']['TranscriptionJobStatus']
        if job_status in ['COMPLETED', 'FAILED']:
            print(f"Job {job_name} is {job_status}.")
            if job_status == 'COMPLETED':
                print(
                    f"Download the transcript from\n"
                    f"\t{job['TranscriptionJob']['Transcript']['TranscriptFileUri']}.")
            break
        else:
            print(f"Waiting for {job_name}. Current status is {job_status}.")
        time.sleep(10)
def main():
    transcribe_client = boto3.client('transcribe', region_name = 'us-west-2')
    file_uri = 's3://DOC-EXAMPLE-BUCKET/my-input-files/my-media-file.flac'
    transcribe_file('Example-job', file_uri, transcribe_client)
if __name__ == '__main__':
    main()

To see more, visit Amazon Transcribe’s Developer Guide.

Both APIs provide streaming transcription. Deepgram offers lower latency, higher accuracy, and easier implementation.

When using Deepgram to transcribe live streaming audio, two of the features you can use to further understand your audio include Endpointing and Interim Results. Both of these features monitor incoming live streaming audio and can indicate the end of a type of processing, but they are used in very different ways:

• Deepgram live streaming looks for any deviation in the natural flow of speech and returns a finalized response at these places. To learn more about this feature, see Endpointing.
• Deepgram live streaming can also return a series of interim transcripts followed by a final transcript. To learn more, see Interim Results.
• If your downstream natural language processing (NLP) requires complete utterances, review our Best Practices for Utterance Segmentation.

Amazon Transcribe can perform real-time streaming through the AWS SDKs, HTTP/2, WebSockets, and the AWS Management Console. Amazon supports FLAC, OPUS-encoded audio in an Ogg container, and PCM (16-bit little-endian, audio format, no WAV). If using HTTP/2 or WebSockets, Amazon requires a format called event stream encoding. Amazon streaming is supported in only 12 languages, and Amazon notes that its real-time streaming speed “may have accuracy limitations in some cases.”

Amazon real-time streaming automatically provides partial results, passing along a results field called IsPartial. If IsPartial is set to true, then the transcripted segment is not complete. If IsPartial is set to false, then the transcripted segment is final. Deepgram has a similar interim results field called is_final. With Deepgram, if is_final is set to false, then the results may be improved. If is_final is set to true, then the transcript is final.

If you require lower latency, Amazon requires you to enable Partial Results Stabilization, which can continually change the text until the segment is final. This feature adds a field called Stable to the results. If Stable is set to true, then the text will not change. If Stable is set to false, then the text may change as the segment is transcribed. Turning on Partial Results Stabilization also affects accuracy, depending on the level of stability required. Higher stability transcribes faster but with lower accuracy.

By default, both Deepgram and Amazon will provide you with the following values:

• transcript
• start_time (duration)
• end_time (duration)
• word (string)
• confidence (float)

Additionally, Amazon and Deepgram provide the following:

We provide sample scripts in Python and Node.js and assume you have already configured either a Python or Node development environment. System requirements will vary depending on the programming language you use:

To get transcripts and customize your response, you can use our SDKs and code samples for:

• Pre-recorded
• Streaming

Each API service has different limitations to ensure performance. Known differences between Deepgram and Amazon Transcribe are detailed below, along with information about how our customers address these differences during the migration process.