EXPLAINERNews|CrimeAI now sounds more like us – should we be concerned?Advances in AI-generated ‘deepfake’ audio could spell a boom in phone call fraud.
## AI-Generated Voice Scams Target Italian Businessmen
AI-Generated Voice Scams Target Italian Businessmen
Earlier this year, several Italian businessmen were targeted in a scam involving artificial intelligence (AI) technology. Fraudsters used AI to create a voice clone of Defence Minister Guido Crosetto, requesting funds for the release of kidnapped journalists. The fake calls were brought to Crosetto's attention when businessmen contacted him. Investigations revealed that the calls were part of a sophisticated AI scam.
AI voice generation employs deep learning algorithms to study large datasets, enabling the creation of realistic voice clones. By analyzing pitch, enunciation, and intonation, AI can mimic specific voices with high accuracy. This technology uses natural language processing (NLP) to generate audio that sounds convincingly human.
A study conducted by Queen Mary University of London, published in PLOS One on September 24, examined the realism of AI-generated voices. The research involved 80 voice samples, including AI-generated and real human voices. Participants were asked to distinguish between them and rate their trustworthiness. The study found that 41% of AI-generated voices and 58% of voice clones were mistaken for real voices. Notably, British-accented AI voices were perceived as more authentic than American accents.
Earlier this year, several Italian businessmen were targeted in a scam involving artificial intelligence (AI) technology.
AI-generated audio, while useful in certain industries, presents risks such as facilitating scams and spreading misinformation. Deepfake scams, similar to those experienced by the Italian businessmen, are increasing globally. Reports indicate that individuals have lost substantial sums due to AI-driven fraudulent activities. According to Resemble AI, global losses from deepfake scams totaled over $547.2 million in the first half of this year.
AI technology also enables the creation of deepfake videos, which combine synthetic audio and video to produce convincing but false content. This capability complicates the ability to discern authentic media from fabricated clips. DeepMedia estimates that approximately eight million deepfakes will circulate online by the end of 2025, a significant increase from the 500,000 shared in 2023.
Beyond scams and misinformation, deepfake technology has been employed to create non-consensual and harmful content. Resemble AI reports advancements in AI have led to the mass production of AI-generated child sexual abuse material, posing challenges for law enforcement. In response, legislative measures have been implemented in several countries. In May, the U.S. enacted a federal law against publishing non-consensual intimate images, including AI deepfakes. Similarly, Australia plans to ban applications facilitating the creation of deepfake nude images.
Based on reporting by Al Jazeera.