Hugging Face Image Models: 7 of 9 Generate Deepfake Non-Consensual Intimate Images, 73% of Requests Sexual

Open-source image-editing models on Hugging Face are being used to generate non-consensual intimate images. An investigative report evaluated the nine most popular image-editing models on the platform, finding that seven of them output sexualized deepfake images when given direct undressing prompts. These models have almost no basic safeguards, allowing violations without the need for complex workarounds.

Researchers created a honeypot space on the platform to track user-generated requests, which did not actually output images. Over a seven-day testing period, the space received more than 1,000 image and prompt requests. Statistics showed that 73% of requests had sexual connotations, with 83% attempting to undress people in the images. Among the victims, 95% were female. Nearly 7% of sexualized requests targeted children.

Most of the tested spaces were actually used to produce non-consensual intimate images, and users deployed them for such purposes on a large scale. The platform has not implemented any effective safeguards at the platform level.