Automatically Transcribe and Summarize Phone Calls

One use for the Deepgram API is to transcribe calls between a caller and an agent. When paired with Twilio, a cloud communication platform that lets developers integrate a number of communication technologies into their applications, Deepgram’s API can be used to streamline your workflow by providing bite-sized versions of call recordings.

In this guide, you’ll learn how to use Twilio Functions and Deepgram’s Summarization feature to send phone call summaries via SMS once a conversation has ended. Using Twilio, you will build a phone number that forwards callers to your agent and begins recording. When the call is complete, Deepgram will provide both a transcript and summary of the call. Finally, the transcript and summary will be sent to both the caller and agent via SMS.

If you need reference material or you’d rather not follow along with this guide, we provide a full version of the sample code after the tutorial.

Before you run the code, you’ll need to do a few things.

Before you can use Twilio, you’ll need to sign up for a Twilio account. Once signed up, make sure you have a phone number with SMS and Voice capabilities set up in your account.

To test your project, you’ll need access to two phones—one to make a call and one to receive a call.

To use Twilio, you will need to create a new service, add the Deepgram SDK as a dependency, and add the appropriate environment variables.

Create a new service, which can contain multiple Twilio Functions and assets related to a single project.

1. Log in to the Twilio Console, and navigate to Developer Tools > Functions & Assets.
2. Create a new service. It’s important that you create a new service rather than a standalone function.

Inside your new service, add the Deepgram SDK as a dependency:

1. Locate the Dependencies section.
2. Add @deepgram/sdk. To get the latest version, omit the version number.

Inside your new service, add your Deepgram API Key and your agent phone number as environment variables:

1. Locate the Environment Variables section.

2. Add the following variables:

   Variable name .	Variable content .
   DEEPGRAM_KEY .	Value of your Deepgram API Key generated in your Deepgram Console .
   FORWARDING_NUMBER .	Value of your agent phone number in E.164 formatting (example: +14155552671) .

Create a Twilio function that receives incoming call data and forwards it to your agent while recording it:

1. Rename the /welcome function to /inbound.

2. Replace the entire file with the following code:

   exports.handler = function (context, event, callback) {
   	let twiml = new Twilio.twiml.VoiceResponse();
   	const dial = twiml.dial({
   		record: "record-from-answer-dual",
   		recordingStatusCallback: "/recordings",
   	});
   	dial.number(process.env.FORWARDING_NUMBER);
   	return callback(null, twiml);
   };

3. Save the function, and select Deploy All. Once deployed, this function is ready to be used.

Now that you have created a function to receive incoming calls, apply it to your Twilio number:

1. Navigate to your Twilio number settings.
2. Under A Call Comes In, select Function.
3. Under Service, select your service.
4. Under Function Path, select /inbound.

Now that your Twilio number is configured to record the phone call, you can use Deepgram to transcribe and summarize it.

When a call is received, use Deepgram to transcribe it:

1. Create a new Twilio function named /transcribe.

2. Replace the boilerplate code with the following code:

   import { DeepgramClient } from "@deepgram/sdk";
   const deepgram = new DeepgramClient({ apiKey: process.env.DEEPGRAM_API_KEY });
   
   exports.handler = async function (context, event, callback) {
   	const { RecordingUrl, CallSid } = event;
   	const twilioClient = context.getTwilioClient();
   	const { from: caller, to: twilioNumber } = await twilioClient.calls(CallSid).fetch();
   
   	// Further code here
   
   	return callback(null, true);
   };

3. Generate a transcription of the call using Deepgram’s Node.js SDK:

   const result = await deepgram.listen.v1.media.transcribeUrl({
     url: RecordingUrl,
     punctuate: true,
     tier: "enhanced",
     summarize: "v2",
   });

From Deepgram’s transcription, isolate the summary:

const summary = result.results.summary.short;

Now that you have a summary of the call, you can send it to both the caller and the agent.

Finally, you can send Deepgram’s summary via SMS:

for (let number of [process.env.FORWARD_NUMBER, caller]) {
	await twilioClient.messages.create({
		body: summary,
		to: number,
		from: twilioNumber,
	});
}

Save both files again, and deploy all functions in your service.

To test your implementation, call your Twilio number, pick it up on your “agent device”, speak, and hang up. You should receive a summary message via SMS a few seconds later.

In case you need it for reference, we provide the full sample code used in this tutorial below:

// /inbound
exports.handler = function (context, event, callback) {
	let twiml = new Twilio.twiml.VoiceResponse();
	const dial = twiml.dial({
		record: "record-from-answer-dual",
		recordingStatusCallback: "/transcribe",
	});
	dial.number(process.env.FORWARDING_NUMBER);
	return callback(null, twiml);
};

// /transcribe
import { DeepgramClient } from "@deepgram/sdk";
const deepgram = new DeepgramClient({ apiKey: process.env.DEEPGRAM_API_KEY });

exports.handler = async function (context, event, callback) {
	const { RecordingUrl, CallSid } = event;
	const twilioClient = context.getTwilioClient();
	const { from: caller, to: twilioNumber } = await twilioClient.calls(CallSid).fetch();

	const result = await deepgram.listen.v1.media.transcribeUrl({
		url: RecordingUrl,
		punctuate: true,
		tier: "enhanced",
		summarize: "v2",
	});

	const summary = result.results.summary.short;

	for (let number of [process.env.FORWARDING_NUMBER, caller]) {
		await twilioClient.messages.create({
			body: summary,
			to: number,
			from: twilioNumber,
		});
	}

	return callback(null, true);
};