Skip to main content

How to Use Speech to Speech (Voice Mirroring) in HeyGen

A
Written by Avi

How to Use Speech to Speech (Voice Mirroring) in HeyGen

Speech to Speech converts a spoken recording into a HeyGen voice while keeping the words and timing of the original performance. If you want your video's voiceover to sound exactly the way you'd say it, with your pacing, emphasis, and pauses, instead of the default text-to-speech delivery, this is how you do it. In the HeyGen app this workflow is called Voice Mirroring and it's part of video creation. Developers can also use Speech to Speech through the API to generate standalone audio and use it in any video they create.

Use Speech to Speech when you want the natural pacing and delivery of a real performance, but need the result in a different voice.

Pricing: The Speech to Speech API is billed at $0.12 per minute of audio. Charges come out of your API credit balance, which is measured in dollars (1 API credit = 1 USD). To learn more about API credits and how to top up, see HeyGen API Pricing & Credits Explained.

Voice Mirroring or the Speech to Speech API?

Voice Mirroring in Studio

Speech to Speech API

Best for

Creating avatar videos in HeyGen

Building speech conversion into another product or workflow, or into any video you create

Input

Record or upload audio in a scene

Send an audio asset, public URL, or base64 audio

Output

Converted voice audio added to the scene and lip-synced to the avatar

A URL for a standalone WAV file

Voice

The voice selected for the scene

An active Instant Voice owned by your workspace

How do I use Voice Mirroring in Studio?

1. Open the audio uploader

Create or open a video in Studio, select the scene and avatar you want to use, then click Upload audio in the script area.

2. Record or upload your performance

Choose one of the following:

  • Record Audio: Select your microphone and record directly in Studio.

  • Upload Audio: Upload an MP3 or WAV file. The current Studio uploader accepts files up to 100 MB.

For the best result, use a clear recording with one speaker and little or no background music.

3. Turn on Voice Mirroring

After you finish recording or select an uploaded file, the Confirm Audio dialog opens. Turn on Voice Mirroring, then click Add audio. The scene uses the voice selected in the Avatar & Voice panel, so set the voice you want there first.

HeyGen transcribes the recording so you can review the words. The selected voice controls the voice identity, while your recording guides the timing and delivery.

How do I convert an existing scene?

If a scene already contains a script, open the scene's ⋯ menu and choose Mirror voice. In the Mirror your voice dialog, the script appears as a teleprompter so you can record or upload a performance for the same scene.

How do I get better results?

  • Use a clean recording with one speaker.

  • Speak naturally and make your pacing, emphasis, and pauses intentional.

  • Start with short segments while you establish the performance you want.

  • Avoid heavy background noise, music, or overlapping speech.

  • Preview the scene before generating the full video.

How do I use Speech to Speech through the API?

The API converts a recording to an active Instant Voice in your workspace and returns a WAV file. It does not create a video or lip-sync an avatar on its own. To turn the audio into a video, see the section below.

Before you start

You need:

  • An API key with the voices:write permission, or an OAuth token with API access

  • The voice_id of an active Instant Voice owned by your workspace

  • A source recording supplied as an asset ID, a public HTTPS URL, or base64 audio

🛎️ Important: Professional Voice Clones are not currently supported by the Speech to Speech API.

Not every voice in your HeyGen voice library is an Instant Voice. To find voices you can use, call GET https://api.heygen.com/v3/models/audio/voices and choose one with mode set to instant and status set to ACTIVE. To create one, send a POST request to the same endpoint with mode set to instant.

Make a request

Send a POST request to:

https://api.heygen.com/v3/models/audio/sts

Example:

curl -X POST "https://api.heygen.com/v3/models/audio/sts" \  -H "X-Api-Key: $HEYGEN_API_KEY" \  -H "Content-Type: application/json" \  -H "Idempotency-Key: sts-example-001" \  -d '{    "voice_id": "YOUR_INSTANT_VOICE_ID",    "audio": {      "type": "url",      "url": "https://example.com/recording.mp3"    }  }'

A successful response includes the generated audio URL and duration:

{  "data": {    "audio_url": "https://files.heygen.ai/generated/speech-conversion.wav",    "duration": 12.48  }}

The request can take up to 150 seconds, so set your client timeout to at least 160 seconds.

Audio requirements

  • Maximum duration: 300 seconds

  • Supported formats: WAV, MP3, FLAC, OGG, M4A, and AAC

  • Maximum size: 100 MB for a HeyGen asset, 32 MB for a URL, or 16 MB for base64 audio

  • Public URLs must use HTTPS, point directly to the audio file, and download within 60 seconds

💡 Pro Tip: Set remove_background_noise to true if the recording needs cleanup. This can improve noisy input, but it also adds processing time.

How do I use the audio in a video?

Speech to Speech works with any video you create, not just Studio. Take the audio_url from the response and pass it into POST /v3/videos as the audio_url to lip-sync it onto an avatar:

curl -X POST "https://api.heygen.com/v3/videos" \  -H "X-Api-Key: $HEYGEN_API_KEY" \  -H "Content-Type: application/json" \  -d '{    "type": "avatar",    "avatar_id": "YOUR_AVATAR_ID",    "audio_url": "AUDIO_URL_FROM_THE_SPEECH_TO_SPEECH_RESPONSE"  }'

What are the retries and common errors?

Use an Idempotency-Key when retrying a request so the same request is not processed twice. If the API returns 409 voice_not_ready, wait for the number of seconds in the Retry-After header, then try again. Requests with the same Idempotency-Key that are retried within 24 hours are not charged twice. Note that when you send an Idempotency-Key, the conversion keeps running if your client disconnects and can still be charged.

Other common responses include:

  • 400 invalid_parameter: Check the voice ID, audio format, duration, and file size.

  • 401 unauthorized: Check the API key or bearer token.

  • 402 insufficient_credit or plan_upgrade_required: Add credits or use an eligible plan.

  • 403: Check the API scope and the member's role.

  • 404 voice_not_found: Confirm that the voice is an Instant Voice in the authenticated workspace. A voice that appears in your voice library but not in GET /v3/models/audio/voices returns this error.

  • 429 rate_limit_exceeded: Slow down requests. The current limit is 30 requests per minute per workspace member.

  • 503 or 504: Retry after a short delay. Use the same idempotency key.

Frequently asked questions

Does Speech to Speech rewrite or translate my recording?

No. Speech to Speech keeps the spoken words and timing. To translate a video, use HeyGen Video Translation.

What is the difference between Voice Mirroring and voice cloning?

Voice cloning creates a reusable voice identity. Voice Mirroring uses a recording to guide the words, timing, and delivery of a selected voice.

What is the difference between Speech to Speech and text to speech?

Text to speech generates audio from written text. Speech to Speech starts with a spoken performance, so the original timing and delivery can carry into the result.

Does the API lip-sync an avatar?

No. The Speech to Speech API returns audio only. To create a lip-synced avatar video, use Voice Mirroring in Studio or pass the resulting audio into a video request with audio_url, as shown above.

Why did I receive voice_not_ready?

The first request for an Instant Voice can require additional preparation. Wait for the Retry-After interval and resend the same request.

Related Articles

Last Updated: October 28, 2026

Related Articles
Did this answer your question?