In the beginning of April, Meta revamped its rules on AI-generated content and deepfakes, and announced the labelling of AI-generated video, audio and images with "Made with AI". In India, Meta platforms has since been flooded with political deepfakes, but the "Made with AI" label is nowhere to be seen.
Political parties have been using AI to target their opponents for months, and appear to have free rein on Facebook and Instagram, with their official and proxy pages riddled with deepfakes, often shared as satire.
Celebrities and politicians have become fair game for memes and face swap videos made with AI, with platforms like Instagram awash with AI-modified content. But ambivalence over AI content by platforms has allowed political parties, and by extension their proxy pages, to pump out deepfake memes targeting their opponents.
Also Read:Explained: How To Know Your Candidate In 5 Steps For 2024 Elections
Ahead of the ongoing Lok Sabha elections, the potential for misuse of AI to deceive the electorate had presented itself as a major concern among fact-checkers, policymakers and election watchdogs. With only one phase left for the conclusion of the polls, deepfakes appear to have been used mostly for political shitposting, with only sparse use for deceptive purposes.
While such deepfakes-led political shitposting escape any fact-checking, average Instagram users are often unaware that the proxy handles pushing out such content act as front for political parties.
Swapping Face, To Win The Race
In February, Congress posted a video on its official Instagram handle where an AI-likeness of Modi appeared to be singing. Weeks later, the Bharatiya Janata Party posted a video where Congress leader Rahul Gandhi's face was swapped with Rashtriya Janata Dal leader Tejashwi Yadav, in a video of the latter giving a speech in the Bihar assembly.
The content becomes more deceptive on proxy pages of political parties.
Paltu Paltan, a surrogate page for the BJP, posted a video of Gandhi with an overlaid audio containing his voice clone where he appears to say that industrialists Gautam Adani and Mukesh Ambani are helping the Congress in states ruled by the party, due to which he is can't speak about them. This video was released in the backdrop of the misleading claim made by Prime Minister Narendra Modi on Gandhi stopping its attack on the alleged favouritism by the government towards these industrialists due to having received support from them. The video simply contained a text saying "Haters will say this is AI".
Yet another video posted by the page showed Aam Aadmi Party supremo Arvind Kejriwal giving a speech. Shared with the same text of "Haters will say this is AI", the video shows Kejriwal, with his voice clone saying he "hasn't lost the habit of doing scams".
Yet another surrogate page for the BJP, Inse Na Ho Payega, made rampant use of face-swapping tools to target opposition leaders.
Looking through the posts by this page, we found videos targeting Congress leaders Sonia Gandhi and Rahul Gandhi, Aam Aadmi Party leader Atishi Singh, AAP leader Arvind Kejriwal's wife Sunita Kejriwal, and Trinamool Congress leader Mamata Banerjee. All of these videos face-swapped on to actors in order to caricature these leaders, while none of them contained any label signifying the use of AI-manipulation.
In a previous investigation, Decode had reported on how both 'Paltu Paltan' and 'Inse Na Ho Payega' are run by a BJP-hired political consultancy firm named Varahe Analytics.
'Badlaav Ki Aandhi', a proxy page of the Congress, posted a video of Modi talking of how the problem of global warming can be solved with 'bhakti' (devotion) towards nature. The video then goes on to satarise Modi's comments, by swapping his face on to those of famous scientists like APJ Abdul Kalam, Albert Einstein, Isaac Newton and CV Raman. This post, too, was lacking any labels to indicate the use of AI.
Viggle Mania
On May 6, Modi quote-tweeted a video that showed a rudimentary AI-generated likeness of him come out on stage and dance. It was a 'Lil Yachty' meme - a video showing US rapper Lil Yachty coming out and stage and dancing, wherein his body and likeness is be replaced by someone else's using an AI tool called Viggle.
Modi's tweet was meant counter to West Bengal Chief Minister Mamata Banerjee’s opposing reaction to a Lil Yatchy meme of her dancing on stage, where she got Kolkata Police to take a sterner action against the user who posted it. That video, now deleted, was made using Viggle.
Based on a 3D foundation model, Viggle boasts of being able to 'mix a character image into a motion video', and 'animating the character with a text motion prompt'. The end result usually appears non-realistic.
Going through the proxy pages of BJP we found many such videos being made using this tool. Examples include a video of Congress leader Sonia Gandhi dancing over a bar counter, Rahul Gandhi dancing under an image of billionaire George Soros, and a video of Arvind Kejriwal dancing with his wife Sunita Kejriwal dancing.
Congress proxy page Congress Express also posted a video of Modi dancing with alleged serial rapist and fugitive Prajwal Revanna.
We also found a Lil Yatchy meme of Kejriwal, and another video ridiculing him, where Kejriwal's likeness is superimposed over that of another man - with both videos being made using Viggle.
Meta's policy on AI-labelling states, "If we determine that digitally-created or altered images, video or audio create a particularly high risk of materially deceiving the public on a matter of importance, we may add a more prominent label so people have more information and context."
However, it is unclear and ambiguous as to what is considered 'high risk', with only a few of the posts flagged in this article - all posted in the context of the ongoing general elections - having a AI disclaimer label.
A month before the polls commenced, Meta stated that only "photorealistic image or video, or realistic sounding audio, that was digitally created or altered to depict a real person as saying or doing something they did not say or do."
It further adds, "It also applies if an ad depicts a realistic-looking person that does not exist or a realistic-looking event that did not happen, alters footage of a real event, or depicts a realistic event that allegedly occurred, but that is not a true image, video, or audio recording of the event."
While most of the deepfakes being shared by political parties and their surrogate pages do not fall under "photorealistic" or "realistic sounding", as the AI tools used to make such content get better with more data inputs, such content is only expected to get more and more realistic.