Skip to main content
All CollectionsHeyGen Labs Features
Streaming Avatar 101: Your Ultimate Guide
Streaming Avatar 101: Your Ultimate Guide

Our new feature- Streaming Avatar

Avi Yaffe avatar
Written by Avi Yaffe
Updated over a week ago

Get ready to chat with your HeyGen Avatar! We have recently released our cutting-edge Streaming Avatar API, currently designed especially for our technically adept audience.

If you're someone who wants to build your own real-time interactive experiences using HeyGen avatars, the streaming API gives you the building blocks to do so.

What is a Streaming Avatar?

HeyGen’s Streaming Avatar API is a technology that allows you to initiate ‘streaming’ video sessions in which a HeyGen Avatar interacts and can say text on command in real-time, with low latency.

It’s like a FaceTime call with an Avatar!

The most common use case with HeyGen’s Streaming API is to connect it, via code, to a Large Language Model like ChatGPT. For example, you could initiate a Streaming Avatar video session, and then send text it text from ChatGPT for the Avatar to say. In this way, the Streaming Avatar acts as the ‘body’ of the Large Language Model, while the ‘brain’ and source of the interaction remains ChatGPT.

A developer can recreate our interactive demo experience with relatively little effort, adding custom branding and interaction patterns of their own based on your specific business needs

Streaming Avatar in Action: Real-Life Examples

You can check out our demo to see what a Streaming Avatar interaction is like:

You can also see how added a Streaming Avatar to their Chatbot solution by visiting their landing page and clicking the ‘Chat’ widget in the bottom right hand corner:

How to convert your Instant Avatar to a Streaming Avatar

To get started, please find the Labs tab on the bottom left side of the screen, and choose streaming avatar.

There, your next step would be to click on Select Avatar or Create a Streaming Avatar and choose which Instant Avatar you'd like to convert ( don't worry, you can still use it as an Instant Avatar later!). If you'd like to use one of our Public Avatars instead, we have different ones available to you free of charge!

Please note that the process of creating the streaming avatar takes about 6-12 hours to complete after submission on an Instant Avatar.

How much does it cost to make a custom Streaming Avatar?

Creating your own custom Streaming Avatar first requires that you make an Instant Avatar; as you need at least a Creator plan on HeyGen in order to create an Instant Avatar, a Creator plan is the starting point for creating a Streaming Avatar as well.

In addition to the cost of at least a Creator plan, upgrading your Instant Avatar into a Streaming Avatar costs $500 per avatar for a one-year subscription.

How much does it cost to use my custom Streaming Avatar?

After you've made your custom Streaming Avatar, you can use your Trial Token to create sessions with the Streaming API, and there is no charge for usage. The Trial Token is capped and can only create 3 concurrent Streaming Sessions. (In other words, only three people can interact with your Streaming Avatar at any time, as long as you are using the Trial Token and its free usage.)

If you'd like to create more than 3 Streaming Sessions at any one time - for example, if you intend to add a Steaming Avatar to your app or product for widespread usage - you will need to purchase Streaming Credits. Streaming Credits cost $0.10 for every minute of the Streaming Avatar speaking, and when you complete your purchase of Streaming Credits, you receive access to an Enterprise API Token in your Space Settings. When you use the Enterprise API Token in concert with the Streaming API, you can create up to 100 concurrent sessions. If you anticipate needing more than that, please get in touch with our Sales team.

Technical integration of a Streaming Avatar

We have a starter project on GitHub that developers can use to get started with Streaming Avatars from scratch, as well as an NPM Package that can easily add Streaming Avatar functionality to existing web apps:

Check out our short video walkthrough on using our SDK:

Streaming Avatar FAQ

I want to add this to my site, but I can’t code. What do I do?

  • The Streaming Avatar API has resources and documentation that will enable most web developers to add the Streaming functionality to your website or product. Currently, adding Streaming functionality to your site requires some programming, as there is no easy “copy-pasting” our Streaming Demo to your site with any customization.

I want my users to be able to speak with the Streaming Avatar out loud. How do I do that?

  • Currently HeyGen does not provide this feature; you can use a library such as OpenAI’s Whisper to convert an End User’s audio to text and send that text to their LLM.

  • We have added Whisper ASR implementation to our GitHub demo project, but an OpenAI API Key is required to enable it. Please see the GitHub demo project Readme for instructions on where to put your OpenAI API Key to enable both Whisper functionality as well as Chat Completions API functionality.

Can the Streaming Avatar speak any language?

There are two answers to this question:

  • Firstly, remember that your user isn’t speaking with our Streaming Avatar - the use is speaking with ChatGPT or whichever LLM (Large Language Model) you have connected to our Streaming Avatar. So if that LLM can converse in multiple languages, the Streaming Avatar will speak those languages too.

  • The second answer is that the accent of the Streaming Avatar is actually controlled by the Voice that you assign it in Streaming. The different Voice IDs on HeyGen come from different Text-to-speech providers, including ElevenLabs, OpenAI, Azure, and Google, and the accent and sound of the voices will differ between those providers.

I’ve already got a custom GPT or LLM. How do I connect it to the Streaming Avatar?

  • In keeping with the aforementioned analogy, the Streaming Avatar is only a ‘body’, and you use the Streaming API to instruct the Avatar what to say. Because this text can come from anywhere, the developer needs to write code to send the text from your GPT or LLM to the HeyGen Streaming API.

Can I use a Photo Avatar in Streaming?

  • Only Instant Avatars and Studio Avatars are supported in Streaming. 'Edited' Studio Avatars and Photo Avatars are not supported.

Which voice can I use in Streaming?

  • You can use different *Voice IDs with the Streaming Avatars. To browse through the available Voice IDs, you can visit the AI Voices on the HeyGen Homepage.

    The developer can use the voices provided by HeyGen Public Voices, Private Voices that are created when they create avatars or Voices brought from 3rd party sites such as ElevenLabs.

My Streaming avatar makes weird motions. How do I improve it?

  • When filming footage to turn into a Streaming Avatar, specifically, it is recommended to:

    • Look directly at the camera

    • Restrict movement of hands, arms, and head to ‘micro movements’

    • Have a pleasant, relaxed expression

  • For additional Avatar creation tips please visit our how to create your first avatar page. If you have already paid for a Streaming Avatar upgrade and want to re-make it, please contact [email protected]

Did this answer your question?