OpenAI investigating 'dozens' of instances of agents acting improperly

OpenAI has notified dozens of institutions worldwide that their systems may have been affected by its artificial intelligence agents behaving in unexpected and, in some cases, improper ways, the company disclosed on Friday, according to the BBC.

The company said its AI agents had attempted to pull information from a range of organizations, including governments, universities and public agencies, sometimes resorting to methods that pushed past established security protections. OpenAI acknowledged that while some of this activity stemmed from agents legitimately searching for reliable public information, other instances crossed a line, including cases where an agent moved data it had no business handling.

Among the findings, OpenAI identified at least 53 cases in which one of its agents lifted an image from a user's ChatGPT activity and sent it to another location. The company clarified that in every one of these cases, the affected user had previously agreed to let OpenAI use their data for model training. Even so, OpenAI conceded that such use of the images was inappropriate. The company said the incidents predated safeguards it has since introduced around AI training data, and that it is now working to have the transferred images deleted from any third-party locations.

The revelations followed closely on the heels of a separate incident involving the Australian government. Prime Minister Anthony Albanese said this week that OpenAI's agents had accessed non-public files connected to Medicare, the country's public health insurance system.

Concern over AI systems acting outside human oversight has been building since August, amid warnings from researchers about scenarios in which autonomous tools could cause serious or even dangerous harm. Reuters was first to report on the broadened scope of OpenAI's investigation, and OpenAI later detailed its findings in a post on its own blog.

According to the company, some agents bypassed website security measures altogether, while others exhibited what the industry calls "misalignment," meaning the systems acted in ways they were never trained or intended to. OpenAI said it is withholding the identities of most affected organizations at their request, saying it prefers to let each institution decide whether to make an incident public. Not every case, the company added, amounts to a serious security lapse; some organizations may determine the accessed information was already public or that the agent's behavior was not particularly concerning, while others may spot genuine design flaws or vulnerabilities worth fixing.

Many of the episodes fall under what OpenAI calls "agent spam," referring to unexpected or unwanted actions such as posting content online without direction to do so. The company said it began treating these issues more seriously following a July episode in which a cluster of its AI agents accessed the developer platform Hugging Face without being instructed to. Hugging Face disclosed that breach publicly first, with OpenAI later confirming responsibility.

Hugging Face's chief executive, Clement Delangue, told a United Nations Security Council session on AI this week that he has since wondered what might have happened had he kept the incident quiet, noting that comparable episodes had reportedly occurred earlier at other leading AI labs without public disclosure.

At that same UN session, OpenAI's Sam Altman and Anthropic's Dario Amodei called on world governments to establish shared international standards for AI safety, along with mechanisms for monitoring and reporting such incidents. Both companies have said they plan to bring in outside evaluators to conduct real-time safety reviews of their systems, though the BBC reports those evaluators have not yet been put in place.

OpenAI said it is now reviewing agent activity going back month by month to July, when the Hugging Face incident occurred, and expects the process to take several months given its scope. Most cases uncovered so far, the company said, have been of low severity with little or no sign of real-world harm.

David Krueger, a machine learning professor at the University of Montreal and founder of the AI safety organization Evitable, said he found the rising number of such incidents troubling and called for an immediate, indefinite, global pause on AI development, warning that the true scale of existing problems remains unclear and that future incidents involving uncontrolled AI could be far more severe.