In October this year, a video of celebrity model Bella Hadid apologising for her past remarks on Israel caught the attention of fact-checkers.
The clip seemed odd as Hadid, who is half Palestinian by descent, has been unwavering in her pro-Palestine stance on her Instagram account where she has nearly 61 million followers.
“It's Bella Hadid. On October 7 2023, Israel faced a tragic attack by Hamas. I can't stay silent. I apologise for my past remarks…” the model appears to be saying in the video.
Turns out the video was a deepfake created using a video of one of Hadid’s old speeches and a voice clone made with AI.
Yishay Raziel, an Israeli sound designer and voice-over artist, who has over 19,000 followers on Instagram, is the creator of deepfakes video targeting anyone famous speaking out against Israel.
Deepfakes are synthetic media that includes images, videos and audio. The ‘deep’ part refers to deep learning - a type of advanced artificial intelligence method that mimics how the human brain learns.
Raziel has also made deepfakes of Queen Rania of Jordan, former adult film actress Mia Khalifa, musician Roger Waters, actor Angelina Jolie, beauty blogger Huda Kattan among others.
“My process involves using RVC for voice cloning and video retargeting for lip sync. I am a voice-over artist myself, so I perform many of the voice tracks myself and replace my voice with the voice models,” Raziel told Decode over email.
Retrieval-based-Voice-Conversion or RVC is an advanced speech-to-speech AI technology that is used in voice cloning.
“It takes approximately 5 hours per video, which includes collecting audio, voice modelling, and syncing the lips,” he added.
Dr. Siwei Lyu - a SUNY Empire Innovation Professor in the Department of Computer Science and Engineering at the University at Buffalo, State University of New York, confirmed to Decode that the videos on Raziel’s Instagram account were AI-generated.
“All the videos on the Instagram accounts have been identified with high confidence by state of the art detection algorithms as AI-generated,” Professor Lyu, a deep fake expert, confirmed over email.
“The algorithms have identified artifacts as
(1) AI-generated audios that do not have the common paralinguistic attributes such as breathing and proper pauses;
(2) lack of synchronisation between audio and video
(3) visual artifacts around the lip area -- these are signs of the use of AI-based lip-syncing models that generate lip shapes to fit input voices. Most notably, teeth show up and disappear in adjacent frames, about 1/30 second apart, which is not physically possible,” Professor Lyu told Decode.
Deepfakes Have Wrongfooted Social Media Platforms
The Israel-Hamas conflict has been a watershed event in the fight against misinformation overwhelming fact-checkers globally with the volume of misinfo and fuelled in part by tweaks to platforms such as Elon Musk-owned X (former Twitter) which monetised its blue-tick verification system while culling its trust and safety teams.
Much of the misinformation about the conflict has stemmed from verified X accounts or has been amplified by them.
The use of generative AI in creating misinformation and disinformation has only made this problem worse.
Deepfakes, which trace their origin to the sleazy world of online pornography, pose an existential threat to democracies. They have been used by big Hollywood studios to de-age actors and show dead actors on screen. However, the democratisation of the technology has also made it prone to abuse. The tech has been abused to generate misinformation, commit fraud, create pornography including child abuse porn, target politicians, journalists, and activists.
A few of Yishay Raziel’s deepfake videos have been tagged as ‘Manipulated Media’ on X and linked to fact-check articles on Instagram. Raziel has verified accounts on X, Instagram and Facebook.
YouTube restricted access to Raziel’s deep fake video of Queen Rania of Jordan in response to a legal complaint from a government entity.
“Regarding the 'Manipulated Media' labels assigned by social media platforms, I fully understand and respect their policies intended to keep viewers informed about the content they encounter, especially with deep fake technology” Raziel told Decode.
“Transparency is paramount,” he added.
“In response to the notice from YouTube about the video of Queen Rania, my work aims to provoke thought and highlight underlying truths, even if they are uncomfortable for some. The discomfort it causes, including the reaction from the Jordanian government, is an indicator of the significant impact and the conversations that need to happen,” Raziel said.
Decode wrote to Meta, X and YouTube to understand how social media platforms are deciding which deepfake videos to allow or disallow on their platforms but is yet to hear back from them.
The pushback from platforms has prompted Raziel to urge his followers to follow him on Telegram where content moderation is almost non-existent.
The Disclaimer Dilemma
Since he first started creating deep fakes around the conflict Raziel has started including a watermark disclaimer of ‘Made with AI’ in his videos in addition to mentioning that the videos were created with AI in the caption of his social media posts.
However, one of the deepfake videos on Raziel’s timeline where a man’s face has been swapped with that of Hamas leader Yahya Sinwar’s face in a video where a gay couple talk about how they met, has no such disclaimer.
“The disclaimer helps but is far from sufficient,” Professor Lyu said.
“Because AI-generated content can be made for innocuous purposes, I think it is OK to have them on social media platforms. However, more effective content moderation approaches should be used to provide the viewers context. They can take steps such as (1) requesting the authors to disclose their use of AI models in creating/editing the content; (2) use screening algorithms to verify if the content is AI-generated; (3) put disclaimer of potential AI generated content before the viewer can watch the content to make sure the viewer is aware of this before watching.” Professor Lyu added.
Meanwhile another verified Facebook user credited Raziel for teaching him sound modelling and creating his first rudimentary deepfake “project”, in sign of the battle fact-checkers are up against.