AI Voice Cloning Tools Vulnerable to Scams: A Consumer Report Investigation

AI voice cloning technology has made incredible progress in recent years, enabling the generation of highly realistic audio from just a brief sample of someone’s voice. While this advancement holds great potential in fields such as marketing, audiobooks, and entertainment, it also introduces significant risks, particularly when misused for fraud and scams. Consumer Reports recently investigated six leading AI voice cloning tools to assess the safeguards these platforms have in place to prevent misuse, particularly in non-consensual voice cloning.

The findings raise concerns about the vulnerability of these platforms to exploitation by scammers. The report reveals significant gaps in protection, and highlights the need for stronger security measures and regulatory oversight in the AI voice cloning industry. Let’s dive into the details of the findings and what they mean for consumers.

Consumer Reports reviewed six major voice cloning platforms: Descript, ElevenLabs, Lovo, PlayHT, Resemble AI, and Speechify. Their primary focus was on the ability of these tools to prevent unauthorized cloning of someone’s voice without consent.

The results were concerning. Four of the six tools — ElevenLabs, Speechify, PlayHT, and Lovo — lacked the necessary safeguards to stop someone from cloning another person’s voice without their knowledge. These platforms only required users to check a box confirming they had the legal right to use the voice being cloned. While this is a legal formality, it offers little in terms of real protection against misuse.

In contrast, Descript and Resemble AI stood out as having stronger measures in place to prevent non-consensual cloning. Descript, for instance, requires users to read and record a consent statement before generating a voice model. This ensures that the voice being cloned belongs to the person who consented. Resemble AI uses a slightly different approach by ensuring that the first voice clone generated comes from a real-time recording. While both methods provide additional layers of security, they are not foolproof. An unscrupulous user could still find ways around these protections by playing previously cloned voice samples or utilizing existing video footage.

The report also pointed out how vulnerable these platforms are to being exploited for scams. One common scam involves cloning the voice of a family member and using it to call another relative, convincing them that an emergency has occurred and money is needed immediately. Because the victim believes they are speaking to a loved one in distress, they are more likely to send money without hesitation. This type of scam is becoming increasingly common and is made more convincing by the advancements in AI voice cloning technology.

Moreover, voice cloning technology has even been used in political manipulation. During the 2024 election, someone cloned the voice of former President Joe Biden to discourage people from voting, spreading disinformation through audio that seemed authentic and trustworthy.

While some platforms like Speechify, Lovo, and PlayHT only required an email and name to create an account, Consumer Reports suggests these platforms should collect more detailed information, such as credit card details, to help trace fraudulent audio back to the individual responsible.

To combat these issues, Consumer Reports put forward several recommendations. These included introducing mechanisms to verify the ownership of voices, such as having users read a unique script, watermarking AI-generated audio to make it traceable, and creating tools to detect AI-generated content. Furthermore, the report proposed preventing the cloning of the voices of public figures or influential individuals and banning audio containing scam-related phrases.

Perhaps the most radical suggestion was to move away from the current DIY model of voice cloning. Instead, Consumer Reports recommended that there should be human oversight in the process, ensuring that voice cloning is done responsibly. They also emphasized the importance of holding companies accountable for the misuse of their technology, in line with Section 5 of the Federal Trade Commission Act, which mandates that businesses must protect their products from harm.

To protect themselves, consumers are advised to remain vigilant when receiving urgent requests for money via phone. If you suspect a scam, it’s crucial to contact the person directly using a separate device and verify their identity before sending any funds.

What Undercode Say:

AI voice cloning has undeniable potential, but this Consumer Reports investigation clearly shows that without proper safeguards, this technology can be a powerful tool for scammers and malicious actors. The report highlights several critical shortcomings in current AI voice cloning platforms, particularly in preventing non-consensual cloning.

One of the major takeaways is the lack of robust, automated security measures in place for preventing identity theft and fraud through voice cloning. While some platforms, like Descript and Resemble AI, have introduced additional steps to ensure that users provide explicit consent, these measures are far from perfect. Cloning a voice without authorization is alarmingly easy on platforms with minimal protection. This reflects a broader issue in AI development, where user convenience and the cutting-edge capabilities of the technology are often prioritized over ethical concerns and security.

The widespread use of AI tools in various sectors, from marketing to customer service, means that the potential for malicious use is high. Fraudulent schemes, like impersonating family members to scam money, rely on exploiting the very trust we place in voice-based communication. This is compounded by the fact that these tools are easily accessible and, in many cases, lack the robust oversight necessary to ensure ethical use.

One of the most pressing concerns is the ability of scammers to bypass the relatively weak verification systems in place. A box that users must check to confirm legal rights to clone a voice is not nearly sufficient to prevent abuse. The suggestion to introduce more stringent verification methods, such as requiring credit card information or making voice cloning more supervised, is a step in the right direction. However, it is clear that the broader industry needs to adopt a more proactive stance in safeguarding against the misuse of AI tools.

Furthermore, the report’s call for watermarking AI-generated content and creating detection systems for deepfakes is timely. As deepfake technology becomes more sophisticated, the public’s ability to differentiate between real and AI-generated audio or video will become increasingly difficult. This makes it essential for both developers and consumers to be more educated about the risks and signs of manipulation.

In summary, while AI voice cloning holds great promise for legitimate applications, there is an urgent need for stricter regulations, more advanced safeguards, and an increased focus on the ethical implications of the technology. The industry should not only focus on the benefits but also on the broader impact these tools may have on privacy, security, and trust in digital communications.

Fact Checker Results:

Descript and Resemble AI: These platforms have introduced safeguards to prevent unauthorized voice cloning, though they are not foolproof.
Fraudulent Use of Voice Cloning: Scams using cloned voices to impersonate loved ones and request money are growing in frequency.
Regulatory Oversight: Consumer Reports advocates for stronger regulations, including the supervision of voice cloning and ensuring companies are liable for misuse under FTC regulations.