Schools told to remove photos of pupils online after threat from AI sexual images

Schools in the UK have been advised by experts to remove photographs of pupils from their websites and social media accounts. This recommendation comes after criminals have used such photos with artificial intelligence tools to create sexually explicit images.

The National Crime Agency (NCA), and the Internet Watch Foundation (IWF), have warned that pictures of pupils can be used by criminals to create child sexual abuse images using AI. These criminals can then demand money from schools to prevent them from publishing the images online.

The IWF has identified at least one unnamed secondary school who has been the target of a blackmail attempt in this way. The blackmailers took photographs of children from the school from the school’s website and social media accounts. They then sent sexually explicit pictures of the pupils, generated using AI, and demanded money or they would publish the pictures. The IWF said that 150 of the images created could be classified as child sexual abuse material under the current laws in the UK. The IWF also said that the incident, which happened towards the end of last year, was not the only blackmail attempt of this kind involving a school that they were aware of.

The Early Warning Working Group (EWWG), an advisory body on online harm, has said that while the problem is not widespread, it is only a matter of time before more schools become targets. The EWWG has recommended that schools only publish images that are harder to misuse, such as photos taken at a distance, from behind, or with blurred images. They warn against images that show pupils face-on and that have identifiable information such as names or faces.

The group also ask whether schools need photographs at all, asking them to consider “whether using imagery without children and young people’s faces can still achieve your objectives”. They advise schools to conduct regular audits of pupil’s photos on websites, social media account, and promotional material.

The Minister for Safeguarding and Violence Against Women and Girls, Jess Phillips, said that this trend was a “deeply worrying emerging threat”. The government have said that they will introduce a ban on AI models which are designed to create child sexual abuse material, and Phillips went on to say that they “will not hesitate to go further if necessary and make sure our laws stay up to date with the latest threats”.

Leora Cruddas, Chief Executive of the Confederation of School Trusts commented on the guidance, saying: “As educators we instinctively want to celebrate children’s achievements and that includes sharing photos and videos of all the good things that go on in our schools – it is deeply depressing that in doing so we potentially have to contend with threats from abusers and scammers”.