The new tool can dub voice into 20+ languages while preserving authenticity & nuances.
Now, you can convert spoken audio or a video with audio in 20+ languages while maintaining the character of the voice. With the new AI dubbing tool by startup ElevenLabs, creators have a new way to reach a wider audience without spending heaving on dubbing artists.
Under the hood, the tool has multiple technologies working in tandem to achieve realistic and practical output, such as voice cloning, synthesis, and text processing. The tool can even differentiate between multiple voices, preserving the nuances of different speakers and ignoring the music or noise.
You can give the tool a spin on the official website and it’s free to start, though you need to pay for the minutes dubbed. In a bid to make it easier for people to localize their content, ElevenLabs has mentioned how the tool can be used to translate and dub videos or convert written narratives into multilingual audiobooks that preserve authentic human voice.
Notably, Spotify has also released a similar tool using OpenAI’s Whisper technology for podcasts while YouTube has teased AI dubbing for creators using the Aloud team, a service from Google’s Area 120 incubator.
Commenting on the concerns surrounding deepfakes, as the tool can easily be misused to generate fake voices of known people, the company said, “While we see our tech being overwhelmingly applied to positive use, we also see an increasing number of voice cloning misuse cases.”
The company mentioned in a tweet that they can trace back the generated audio back to the user but are looking for more ideas, including additional account verification, copyright verification, and even potentially manually verifying each request.
