Reports of deepfake image-based abuse in Australia have surged, doubling over the past 18 months. At least one incident has been reported each week in schools across the country. These statistics were released by Australia’s eSafety Commissioner on Friday, following an investigation into explicit images targeting female students at a high school in northern Sydney.

Commissioner Julie Inman Grant confirmed that her office is collaborating with New South Wales Police and the NSW Department of Education regarding the distribution of these digitally altered images in online groups. "We've put out a deepfake image-based abuse incident management tool for schools so they know when to go to police and then they know when to come to us to have the content taken down," Inman Grant stated.

Several families from the affected school visited Eastwood Police Station recently, expressing concerns that the school had not adequately addressed the issue. Inman Grant noted that incidents occurring in schools represent only a small fraction of the total reports received. "We are seeing deepfaked image-based abuse incidents happening at least once a week in Australian schools. This is a real cause for concern. This is really putting potential online harms on steroids," she said.

The rise in deepfake incidents is not limited to Australia. Colm Gannon, chief executive of the International Centre for Missing and Exploited Children, highlighted a significant increase in the United States. He reported a staggering 1,325 percent rise in AI-generated material and deepfakes affecting children globally. "That escalation is to such an extent that we have the federal government passing legislation," Gannon said. He noted that various states and territories in Australia are also working on legislation to combat these issues.

Inman Grant believes Australia is at the forefront of addressing this trend and is in discussions with international partners to implement further restrictions. "We are taking some action today against some 'nudifying' services and in this case against a company that probably makes some of the most popular undressing apps that are used by at least 100,000 people in Australia," she explained.

Working with the UK’s Ofcom, Inman Grant reported a 98 percent success rate in removing harmful deepfake images from online platforms.

NSW Women's Safety Commissioner Hannah Tonkin shared her concerns after speaking with high school female students who expressed fear of being targeted. "Many are seeing it happen to their friends and they're really scared it's going to happen to them," Dr. Tonkin said. She criticized 'nudify' apps as "disgusting technology" that dehumanizes and degrades women. "These types of incidents can have devastating impacts, particularly on women and girls who are overwhelmingly the targets," she added.

Acting NSW Minister for Education Courtney Houssos emphasized the need for collaborative efforts to tackle this issue. "We need to work together to address this scourge for our students, and sadly for some of our teachers," she said. Houssos mentioned that the NSW Department of Education is partnering with the eSafety Commissioner to develop digital literacy education for students, aiming to better prepare them for navigating the internet and AI responsibly.