AI-generated deepfake videos, voice cloning emerge as potential threats during poll season

New Delhi: As political parties go all out to woo voters ahead of the general elections, cybersecurity experts have raised concern over the possible misuse of artificial intelligence and deepfake technology, among others, to influence the electorate.

Lok Sabha polls in India will take place in seven phases between April 19 and June 1.

The Election Commission of India has already issued standard operating procedures for identification and quick response to fake news and misinformation.

"Deepfake videos and voice cloning are two tools that could be massively used during the election campaign," a senior officer of the Delhi Police's cyber crimes unit said.

He believes that a major challenge before the police is timely detection of such content and taking prompt action.

There is no technology available that can automatically detect and differentiate between original and fake video content created using artificial intelligence (AI).

"By the time it gets noticed, the damage is already done because it gets spread on social media," the officer said.

In January 2024, during the New Hampshire primary of the Democratic Party in the US, a robocall mimicking President Joe Biden's voice falsely advised voters not to participate, claiming it would affect their eligibility for the general election.

In Slovakia, an AI-generated voice, mimicking that of a liberal candidate, discussing plans to raise alcohol prices and rig the election was widely circulated on Facebook.

Similarly, a manipulated audio clip falsely implicated a presidential candidate in plans to manipulate ballots during the Nigerian elections of February 2023.

In Bangladesh, deepfake videos of opposition politicians Rumin Farhana in a bikini and Nipun Roy in a swimming pool surfaced on social media ahead of the national elections.

Former Indian Police Service (IPS) officer and cybersecurity expert Triveni Singh said the widespread dissemination of AI-generated misinformation can erode public trust in the electoral process and democratic institutions.

The government should engage with stakeholders, including cybersecurity experts, tech companies and civil society organisations, to develop transparent and fair guidelines for evaluating and approving AI tools, Singh said.

Talking about the measures being taken to tackle the potential danger that AI poses during elections, he said the Centre has made the Indian Cybercrime Coordination Centre (I4C) the nodal agency for taking down objectionable online content.

As soon police from any state inform the I4C about malicious content online, they will reach out to the social media companies to get the content deleted, Singh added.

Cyber crime units of state police are also patrolling online to identify fake news or inappropriate content, he said.

Singh suggested that security agencies can create specialised teams equipped with tools to detect and analyse deepfake and voice cloning. These teams should be available round the clock, especially during critical periods such as elections.

He also urged the authorities to conduct awareness campaigns to educate voters about deepfakes and voice cloning and on ways to critically evaluate information on social media.

Shashank Shekhar, co-founder of Future Crime Research Foundation, a think-tank incubated at IIT-Kanpur, said law enforcement agencies work closely with social media companies to detect and remove malicious content.

Promptly identifying AI-generated content or synthetic video/audio remains the biggest challenge, he said.

AI-powered misinformation campaigns can influence voter behaviour by spreading false narratives or amplifying divisive content, he said.

Detecting the source of the AI-generated content is another challenge. Many foreign countries have an interest in Indian elections and the possibility of large-scale manipulations from these countries cannot be ruled out, Shekhar said.

According to an Election Commission official, social media cells have been constituted in collaboration with cyber cells in all districts for quick response and action within the legal framework.

The commission has requested all political parties and candidates to ensure that their supporters do not spread fake news. Strict watch on social media is being kept by the poll body's Media Certification and Monitoring Committee to ensure that the election atmosphere is not vitiated, he said.