ElevenLabs AI Voice Generation Linked to Russian Influence Operation

The Emerging Role of Generative AI in State-Sponsored Influence Operations

Generative AI, while offering numerous benefits, has demonstrated a susceptibility to misuse. Instances range from the fabrication of academic research to the unauthorized replication of artistic styles.

Recent evidence suggests a new area of concern: the utilization of this technology in state-level influence campaigns.

“Operation Undercut” and the Use of AI Voice Generation

A recent report by Recorded Future, a threat intelligence firm based in Massachusetts, indicates that a specific campaign received “very likely” assistance from commercially available AI voice generation products.

This includes technology developed by ElevenLabs, a rapidly growing startup in the AI space.

The campaign, known as “Operation Undercut,” was designed to weaken European support for Ukraine. It heavily featured AI-generated voiceovers in fabricated or misleading video content.

Targeting European Audiences with Disinformation

These videos, aimed at audiences across Europe, disseminated narratives critical of Ukrainian political figures.

They also questioned the efficacy of providing military aid to Ukraine, among other themes.

For instance, one video asserted the ineffectiveness of defenses against modern weaponry, specifically stating that “even jammers can’t save American Abrams tanks,” thereby suggesting the futility of supplying advanced armor to Ukraine.

Researchers at Recorded Future believe the video creators “very likely” employed AI voice generation, including ElevenLabs’ technology, to enhance the perceived authenticity of their content.

Verification was achieved by submitting the audio clips to ElevenLabs’ own AI Speech Classifier, which is designed to detect audio created using their platform, resulting in a positive match.

ElevenLabs’ Response and Further Observations

ElevenLabs did not provide a comment when contacted regarding this report.

While Recorded Future identified the probable use of multiple commercial AI voice generation tools, only ElevenLabs was specifically named.

Interestingly, the campaign’s organizers inadvertently highlighted the benefits of AI voice generation by releasing some videos with genuine human voiceovers that exhibited “a discernible Russian accent.”

In contrast, the AI-generated voiceovers were delivered in several European languages – including English, French, German, and Polish – without any noticeable foreign accents.

The use of AI also facilitated the rapid dissemination of the misleading clips in multiple European languages, such as English, German, French, Polish, and Turkish, all of which are supported by ElevenLabs.

Attribution and Limited Impact

Recorded Future attributed the operation to the Social Design Agency, a Russia-based organization sanctioned by the U.S. government in March for maintaining “a network of over 60 websites that impersonated genuine news organizations in Europe.”

These spoofed websites were then amplified through fabricated social media accounts, spreading misleading information “on behalf of the Government of the Russian Federation,” according to the U.S. State Department.

The report concluded that the overall impact of the campaign on European public opinion was minimal.

Previous Instances of Misuse and ElevenLabs’ Safeguards

This is not the first time ElevenLabs’ technology has been implicated in alleged misuse.

In January 2024, a robocall impersonating President Joe Biden, intended to discourage voters from participating in a primary election, was traced back to the company’s technology, as determined by a voice fraud detection firm, according to Bloomberg.

In response, ElevenLabs implemented new safety features, including the automatic blocking of voices resembling politicians.

ElevenLabs explicitly prohibits “unauthorized, harmful, or deceptive impersonation” and employs a combination of automated and human moderation to enforce this policy.

Rapid Growth and Investment

Since its establishment in 2022, ElevenLabs has experienced substantial growth.

The company’s Annual Recurring Revenue (ARR) has increased from $25 million to $80 million in under a year, and its valuation is projected to reach $3 billion, as previously reported by TechCrunch.

Notable investors include Andreessen Horowitz and former Github CEO Nat Friedman.