Audio deepfakes, or AI-generated audio that impersonates someone, aren’t new — and they’re the reason that OpenAI, the company that brought AI chatbots to the mainstream, is holding back on releasing its latest offering.
OpenAI announced on Friday that it was choosing to preview, but not broadly release, text-to-speech realistic voice generator Voice Engine because of “the potential for synthetic voice misuse.”
In a blog post, the company outlined Voice Engine’s ability to take a 15-second sample of someone’s voice and emotionally and realistically mimic it as directed by a text input.
“If you have the right audio setup, it’s basically a human-caliber voice,” Jeff Harris, a product lead at OpenAI, told Bloomberg. “It’s a pretty impressive technical quality.”
Related: Deepfake Scams Are Becoming So Sophisticated, They Could Start Impersonating Your Boss And Coworkers
OpenAI has been privately testing Voice Engine since developing it over a year ago and has identified that it can be “used for good” in its blog post.
One application that the company previewed, Voice Engine supports people who are non-verbal by giving distinct voices across many languages. Livox, an alternative communication app, has already started using Voice Engine for that purpose, according to OpenAI.
Voice Engine could also translate videos and podcasts into other languages and provide reading assistance to children and non-readers with audio content.
Related: Tennessee Just Passed a New Law to Protect Musicians From a Growing AI Threat
OpenAI pointed to its AI safety and voluntary commitments policies when stating its rationale for previewing, and not releasing, Voice Engine to the public. The preview was meant to showcase Voice Engine’s capabilities while also emphasizing “the need to bolster societal resilience against the challenges brought by ever more convincing generative models,” the company stated.
Synthetic voices have captured interest in the startup world, with AI voice cloning company ElevenLabs valued at $1.1 billion in January after launching in beta only about a year ago. The technology has also come under fire for the new power it gives cybercriminals, who can use it to impersonate people or access funds through voice-based banking.
OpenAI previewed an AI video generator called Sora last month that creates realistic videos from prompts.
Related: ‘This Is a Serious Problem’: Mr. Beast Slams AI Deepfake
