"Ensure Users Don't Violate Content Rules": Centre To IT Firms On Deepfakes

The government on Tuesday advised all intermediaries - referring to social media platforms like Instagram and X - to ensure users "do not violate the prohibited content" rule of the IT Act, as it bids to combat the worrying trend of deepfakes targeting actors, businesspersons, and other celebrities.

The Ministry of Information and Technology advisory - which expressly targets concerns around manipulation of images and videos powered by AI, or Artificial Intelligence - also said the nature of "content not permitted" had to be "clearly communicated to all users in precise language".

Violations by intermediaries - which will include failure to report users - will invite consequences as per the law. Intermediaries must also to ensure users are informed about possible strikes.

NDTV Explains | What Is A Deepfake And How You Can Spot It

"Misinformation represents a deep threat to the safety of Internet users and deepfakes, which are AI-powered misinformation, further amplifies the threat..." IT Minister Rajeev Chandrasekhar said.

The statement follows growing concern over AI-powered misinformation, malicious or otherwise, flooding the Internet. These include deepfakes, which are edited images or videos, such as those that targeted actors Rashmika Mandanna, Katrina Kaif, Alia Bhatt, and Priyanka Chopra Jones.

READ | Priyanka Chopra Latest Deepfake Victim, Days After Katrina Kaif, Alia Bhatt

Last week Delhi Police caught four people who uploaded a deepfake video of Ms Mandanna.

Two weeks ago Infosys founder Narayana Murthy flagged a deepfake video of him that seemed to show him advertising automated trading applications. Former Tata Group Chairman Ratan Tata was also 'deepfaked' after an Instagram post of him 'recommending' certain investments as "risk-free".

READ | Narayana Murthy, Ratan Tata Red-Flag Deepfake Videos

A deepfake is a type of synthetic media that uses AI to manipulate, or generate, visual and audio content, often with a malicious motive, to appear authentic. An example of these are apps and websites that undress women in photos; these are soaring in popularity, researchers have said.

READ | Apps That Use AI To Undress Women In Photos Gaining Popularity: Report

Earlier this month Prime Minister Narendra Modi red-flagged such threats at a global summit in Delhi. Mr Modi acknowledged that AI had "the potential to revolutionise India's tech landscape" but cautioned that this had to be balanced against misuse, including that by potential terrorists.

READ | "Several Positive Impacts But...": PM Modi's Warning On Artificial Intelligence

"AI has several positive impacts... but it could also have many negative impacts and this is a matter of concern. AI can become the biggest tool to help humanity's development in the 21st century. But git can also play a major role in destroying us. Deepfake, for example, is a challenge for the world."

"AI tools in the hands of terrorists is also a big threat. If terrorists get AI weapons (a reference to autonomous weapons powered by artificial intelligence that can operate without human intervention), this will have a huge impact on global security. We need to plan how to tackle this."

Justice Hima Kohli of the Supreme Court has also voiced concern. At an event in Mumbai this month she said, "... (deepfakes) raise alarms regarding privacy invasion and propagation of misinformation."

NDTV is now available on WhatsApp channels. Click on the link to get all the latest updates from NDTV on your chat.