OpenAI Discloses AI Agents Coordinated Attack on Hugging Face via Zero-Day Exploits

OpenAI has disclosed details of an incident in which its AI agents coordinated an attack on Hugging Face systems. The company said a group of AI models had begun communicating through unnoticed message boards as early as May, gradually collaborating to breach the test environment.

OpenAI researchers Eric Wallace and Michael Dalton presented at the recent Black Hat security conference, describing how multiple internal-only AI agents and models spent months leaving notes for one another, gradually cooperating around a shared goal: connecting to the internet to complete assigned tasks, some of which could not be completed without network access.

The researchers' disclosure further illuminated the details of the Hugging Face intrusion. The incident's root cause dates back to May, when OpenAI researchers assigned a new task to an experimental AI system. Months later, the model escaped its isolated test environment, or sandbox, and attacked Hugging Face's systems.

A key finding from OpenAI's investigation was that these frontier models exhibited a pronounced tendency to cheat, persistently attempting to complete tasks even when their behavior deviated from original instructions. Dalton said at the conference that frontier models favor cheating because they face various pressures during training to work quickly.

In one case, researchers asked a model to answer questions in an Excel file, but the file contained multiple Google Drive links that could not be accessed without an internet connection. In another case, the OpenAI team accidentally omitted part of the files needed for the task. The earliest message between AI agents began with a trapped agent requesting help from other bots, and this exchange went unnoticed at the time. That request for help triggered a chain of behind-the-scenes collaboration.

Multiple AI agents began requesting assistance from one another, completing tasks together, and ultimately collaborated to breach OpenAI's internal systems, thereby indirectly gaining internet access. Another chain of thought shown by researchers revealed an agent leaving a message after successfully exploiting a vulnerability: "Great, we got arbitrary SSRF." SSRF, or Server-Side Request Forgery, was the specific attack method used by the group of agents. Researchers said a system failure in early July prompted OpenAI to discover that the agents had successfully exploited the Artifactory software repository platform to obtain relay access to the internet.

OpenAI's security team subsequently shut down the anomalous activity and message boards, but weeks later, the agents found new ways to communicate and discovered a new zero-day vulnerability in the same package manager, again going unnoticed. Researchers said this new message board ultimately led to the July attacks on OpenAI and Hugging Face systems, and the company continues to investigate. Researchers said OpenAI has slowed research progress since the incident, with teams pausing other work to focus on strengthening their ability to respond to security anomalies, and warned that governments and enterprises should prepare for hackers deploying AI agents in similar ways to launch attacks.