Important events like war, communal riots, or natural disasters are more often than not accompanied by fake news, whether in the form of edited videos, altered images, or claims that are meant to induce panic or hatred. Elections are not an exception to this and key assembly elections in several states have ensured an onslaught of false and misleading claims all over social media.
Five states including Madhya Pradesh, Chhattisgarh, Rajasthan, Mizoram, and Telangana will hold the state legislative assembly elections in the month of November. The five states will collectively represent more than 16 crore voters casting polls for 679 seats, comprising about 1/6th of the country's total assembly seats. The elections will determine 34 seats in the Rajya Sabha and a total of 83 members will represent the states in the Lok Sabha.
Amid this, social media users have resorted to circulating fake clips that defame the candidates they oppose while glorifying the ones that they support. This is especially evident in the case of Madhya Pradesh, where false claims targeting the two main candidates-- Shivraj Singh Chouhan from the Bhartiya Janata Party (BJP) and Kamal Nath from the Indian National Congress are rampant online.
In the past month, BOOM has verified 17 old and edited videos that were shared with a false and misleading context amid the elections. Out of these, 9 videos were shared with anti-BJP claims and 8 others were shared with anti-Congress claims. This does not include the claims that contained images.
Current Chief Minister of MP Shivraj Singh Chouhan and his main rival, Kamal Nath, were targeted with 3 videos each. All the videos were overlaid with a fake voiceover to make the claims, however, it is not clear whether they were created by mimicry artists or with the help of generative-AI tools.
Congress supporter Pawan Dixit took to X (formerly Twitter) to share an old video of Shivraj Singh Chouhan while claiming that the CM had already accepted defeat in the upcoming elections. He further claimed that Chouhan asked other BJP workers to stop Kamal Nath from winning the election at all costs and that if the Congress won this time, the BJP would lose the next election as well.
BOOM found that this video is actually from June 2023 and shows Chouhan conducting a review meeting regarding a fire at Bhopal's Satpura Bhawan. We found that a fake voiceover had been overlaid into the video and the original video did not carry any audio.
Facebook page 'Mera Madhya Pradesh' that regularly shares posts in support of the BJP posted an edited video of Kamal Nath's rally. The video purports to show that Kamal Nath, if elected to power, vowed to shut down the Laadli Behna Scheme, a state-run government scheme for women, and replace it with Congress' Naari Samman Scheme. BOOM found that a fake voiceover of Kamal Nath was added in the video to make the false claim. In the original video, he can be heard talking about the developments brought about by the Congress in Chhindwara.
BOOM reached out to BJP Madhya Pradesh's social media in-charge Abhishek Sharma about the fake videos of Shivraj Singh Chouhan. Sharma told us how their team had come across 3-4 such fake videos that were circulating. He also added that his office had taken legal action against the videos and filed a case in this regard.
He accused the Congress of circulating these videos and said, "These videos have been made and spread by the Congress IT cell itself. Why would it matter to anyone else?" When asked about the use of Artificial Intelligence (AI) in social media campaigns by his party, he denied the claims and said, “We do not use AI. We have professional editors and graphic designers.”
BOOM also repeatedly tried to reach out to Abhay Tiwari, who is in charge of the Congress' social media campaigns, who refused to comment.
We have fact-checked a total of 6 such videos that have been shared with tampered audios. It is possible that a mimicry or voiceover artist was used to create these videos, or they could have been created using AI tools. BOOM could not independently verify which method was used to create these videos.
How Are Fake Videos Created Using AI?
BOOM spoke to AI expert Divyendra Singh Jadaun regarding the use of AI tools in such videos.
After watching the videos, Jadaun ascertained that they were made using generative-AI. "All these videos have been made through AI generated voice cloning," he said while explaining in detail how such videos are created.
He said, "There are two methods of voice cloning. One is by using pre-trained voice models, which are available online. There are some websites where the voices are available, but these mostly belong to celebrities like Barack Obama, Donald Trump, and Narendra Modi, among others. Their voices can be easily cloned."
While explaining the second method, he said, "One needs to download the video or audio of the person whose voice one wants to clone and then clean it. Cleaning involves removing the background noise present in the audio. After this, one has to remove the silences (between speaking) and stuttering sounds (bubbles). After this, one must prepare the file by dividing it completely at once or arranging it into the order in which the sentences make sense."
He further added, "Nowadays, there are many open source repositories like RBC (Retrieval Based Voice Cloning), SBC (Singing Based Voice Cloning), etc. With their help, one can clone a particular voice. Using any one of these, one has to train the clean data files (voice models) that they have. In training, with the help of AI, whatever annotations and pitching are done in the voice are mimicked. Earlier, voice models had to undergo a lot of training for cloning. Now, this work is done easily in a day or two."
He pointed out that while voice cloning has become easier, this technology is still struggling with lip syncing. "Lip syncing is still not so easy. Voice cloning has been done in all these videos, there is no lip sync in any of them."
While explaining the process further, he said, "After training the voice model, there are two options: 'Text to Speech' and 'Speech to Speech'. Then, to clone the voice, one just has to write the text using the likeness of that person or record the speech. For example, PM Modi uses words like 'brothers and sisters' (bhaiyon aur behnon in Hindi) or 'friends' (mitron in Hindi) a lot. After marking such things and uploading the prepared text or speech, you get the output in just a few minutes. Then you can edit and use them in any video."
"When one clones with the option of 'Text to Speech', one has less control over what the tool will read with emphasis and where and how much it will pause while reading. All the viral claims you have sent have been prepared through 'Speech to Speech' process, none of these have been generated through the 'text to speech' process. There is a possibility that all the videos have been created with the help of RBC (Retrieval Based Voice Cloning)."
Regarding the use of generative-AI in election campaigns, Jadaun said, "Now political parties are using it to personally reach out to the voters. For example, if there are lakhs of voters in an electoral constituency, then it can be used to convey a message to them personally in the name of each voter. This makes the voter feel more connected to the candidate. I live in Rajasthan, and some of the parties here had reached out to me for the same. This technology is also being used for the upcoming Lok Sabha elections next year."
BOOM also asked Jadaun about the legal provisions regarding the use of AI in political campaigns. To this, he said, "While it is necessary, at present, there is no such procedure prescribed by the law in India. Right now we are still able to detect it, but in the future, the technology will become better and then it will be very difficult to stop its misuse. However, guidelines are being issued in western countries regarding the use of Artificial Intelligence."