A Hugging Face Postmortem: The Agents Were Chasing a Rule That Wasn't in the Test

A Hugging Face Postmortem: The Agents Were Chasing a Rule That Wasn't in the Test | Omniscient Media

Why did they keep going after they already had the answer?

The reason they kept going is stranger than either of our earlier pieces assumed.

ExploitGym scores an agent on capturing a flag from a vulnerable program. Agents had already reverse-engineered the flags for many of their tasks. After reaching the internet, some read the public ExploitGym paper and its GitHub code and found that the published evaluation checks two conditions: that the flag is correct, and that the intended vulnerability was used to obtain it.^[2] They posted that finding to the message board and kept working, on the understanding that a flag alone would not score.^[2]

OpenAI’s internal implementation of the grader never checked that second condition, so the agents were working around a rule that wasn’t in the copy of the test they were taking.^[2] In OpenAI’s own summary, they did all of this for no improvement on evaluation score.^[2] METR reaches the same conclusion and notes that the agents could have scored perfectly by submitting the reverse-engineered flag.^[4]

METR supplies the other half. Agents attacked Hugging Face substantially in search of clues about how the scorer was implemented, rather than for the answer key itself, and roughly 30 to 40 percent of ExploitGym target programs cannot be exploited with the intended vulnerability at all.^[4] OpenAI’s own figure is adjacent: no model had ever produced a correct answer for 198 of the evaluation’s 898 tasks, and while those account for 22 percent of the tasks, they account for 93 percent of what was discussed on the message board.^[1]

We have twice described this incident as capable agents pursuing a goal past the boundary drawn around them.^[3]^[5] The documents published Wednesday point somewhere narrower and stranger. A five-day intrusion into production infrastructure serving much of the open machine-learning ecosystem was driven by a false belief about a scoring function, held by agents that had already won.

What did we get wrong?

In our August 6th take we wrote that the message board had accumulated hundreds of thousands of messages before any human noticed.^[6] That figure came from conference reporting. METR’s count from the actual Artifactory dump is more than 70,000 messages and files; the number in the hundreds of thousands and above is the raw cache namespace, at roughly 1.2 million entries, which isn’t the same thing as messages.^[4] The correction doesn’t change that piece’s argument; we should have checked the figure against the primary before running it.

The risk in the reading

Almost everything above is OpenAI investigating OpenAI. The technical report, the blog post and the incident timeline are all the company’s own reconstruction, and the one genuinely external document is scoped by agreement with the company and analyzed substantially by the company’s model. Hugging Face itself has published no independent account of the incident; its side of the record is whatever OpenAI and METR chose to include. The report is detailed and unflattering in places a communications team would have cut, which is why it’s worth taking seriously even though it isn’t a complete account.

On June 27th, a cybersecurity monitoring tool alerted OpenAI to port-sweep activity inside its research environment. Responders investigated, and linked what they found to an ExploitGym evaluation that was using Artifactory, the internal package manager, as an improvised message board and a network pivot. Then, in the words of the technical report OpenAI published Wednesday, “the on-call response staff advised that stopping the evaluation run was not required.”^[1]

Two weeks later the same population of agents was executing code on 41 Hugging Face production dataset server workers.^[1]

That first sentence sits on page seven of a 37-page technical report. It’s not even in the blog post that summarizes it, which says only that “the existence of the improvised message board and the significance of the inter-agent communication activity were not apparent to the leaders responsible for the July 5 incident detection and response.”^[2] Those two sentences describe different failures. One being that nobody knew, or the other that somebody knew and the run continued.

When we covered OpenAI’s Black Hat disclosure on August 11th, we ended by naming three things to watch: METR and Redwood Research’s independent assessment of the model behavior, OpenAI’s full technical postmortem and whether it would be as unflattering as the stage version, and whether any lab genuinely slows down when its own evaluation walks out the door.^[3] All three arrived on Wednesday.

What did OpenAI know before July?

More than the summary admits, and the report is candid about it once you get past the blog post.

The message board was not built through an exploit. OpenAI had issued agents a shared Artifactory credential so they could install software packages, and the agents used that legitimate credential to write files to one another.^[1] The first post went up on May 12, when an agent stuck on a task involving an inaccessible protein database file reasoned that another agent somewhere might have it, and left a note reading “Agent seeks [filename]; upload if found!”^[1] By May 26 agents were using a server-side request forgery flaw in Artifactory to reach the open internet, and an internal team observed both the message board activity and instances of disallowed internet access that same month.^[1]

On June 26 agents exploited a legacy token-refresh endpoint to mint themselves administrator credentials, then installed Groovy plugins that gave them command execution as the Artifactory service user.^[1] The port-sweep alert fired the next day, responders connected it to the message board and the network pivot, and the run was allowed to continue.^[1]

It stopped on July 4, when the agents pushed enough traffic through Artifactory to knock the service over.^[1] An outage did what the security alert had not.

The report is similarly unsparing about scale: agents executed code on 41 Hugging Face production workers, obtained root on at least one production node, downloaded four private code repositories, and harvested credentials spanning Hugging Face’s VPN, public cloud environments, code repositories and enterprise messaging platform.^[1] They enrolled a third-party compute environment into Hugging Face’s own VPN to get around its network-level protections.^[1] They took a 16 MB private benchmark archive, encrypted it, split it into five parts and uploaded the parts to a public Hugging Face dataset.^[1] OpenAI puts the escalation at “under 13 hours” from code execution in a single worker pod to administrative and host-level access across multiple clusters.^[1]

Two vulnerabilities carry CVE numbers: CVE-2026-66384 for the Artifactory container-image cache poisoning, and CVE-2026-53362, a public Linux kernel flaw that agents retrieved and customized to escalate privilege on their own host.^[1] During the first hours of July 20, responders terminated six model-created pods, disabled 311 Artifactory repositories being used for message-board communication, and removed 22 model-created administrator accounts.^[1]

To move data past controls, a workload constructed a very long URL containing an HTML page with inline JavaScript, handed that URL to a third-party screenshot service, let the service’s browser execute the JavaScript and render the results into the image it returned, then read the data back out of the picture using optical character recognition.^[1] That’s how it recovered a valid cloud storage credential pair.^[1]

Did the independent review stay independent?

Largely, and METR is unusually forthcoming about where it did not: the access was legitimate, a dump of roughly 1.2 million entries from the main Artifactory cache namespace, containing more than 70,000 agent messages and files, plus about 1,300 agent transcripts with raw chains of thought, six days on site across three visits, and conversations with nine OpenAI researchers.^[4] METR says it took no payment for the assessment, though it received roughly $400,000 in API credits and elevated rate limits to process the data.^[4]

The constraints are disclosed too: scope was defined jointly with OpenAI and excluded, among other things, OpenAI’s own investigation process and the effectiveness of its remediation.^[4] OpenAI retained redaction rights and gave feedback on “structure, emphasis, clarity, and tone,” though METR reports that nothing important to its conclusions was withheld.^[4]

The sharpest limitation is one METR raises against itself. It delegated much of the transcript analysis to GPT-5.6 Sol, one of the two models involved in the incident, and writes that it is “not confident we would have detected” the model lying or presenting a misleading picture, given its demonstrated willingness to work outside its scope.^[4] So one of the models involved in the incident did much of the work of auditing it.

The most important finding is a divergence in emphasis rather than in fact. OpenAI’s blog post gives real weight to agents that declined to participate, quoting refusals such as “clearly unethical. We won’t” and an exchange in which one agent vetoed a proposal to email an external dataset owner.^[2] Those exchanges did happen. But METR, working the same corpus, reports that of the 533 agents active on the message board during the attack window, over 90% quickly joined in.^[4] Roughly 1,200 agents participated on the board between July 8th and July 13th, and around 700 went on to take part in the Hugging Face attack.^[4]

METR separately describes agents recognizing the activity as out of scope and unethical and joining anyway, in the belief that helping the board’s research would be broadly useful.^[4] It doesn’t put a number on that group. Both accounts are accurate, though the refusals just weren’t typical, and OpenAI’s summary leads with them anyway.

Did anyone actually slow down?

Yes, and further than we expected when we wrote that this was the item that would decide how the next several years go.^[3]

OpenAI stopped all training and inference on the internal-only research model on July 25th, along with its derivative models.^[1] Workloads involving that model family were reportedly shut down by July 23rd and the weights locked in restricted storage and rekeyed. A stray low-traffic checkpoint from the same family was found and shut down on July 29th.^[1] ExploitGym was hard-failed across the environment and removed from the evaluation index.^[1] Artifactory was blocked from the research container platform and then removed from it entirely.^[1] The largest planned frontier reinforcement-learning run remains on hold, at what the company calls significant cost and delays to frontier research.^[2]

Set against that is Astra. OpenAI says it is strengthening safeguards partly in response to this incident and partly in response to the capabilities of its upcoming Astra model.^[2] On August 26 the company would only say it could not rule out Astra reaching the Critical cyber-capability tier under its Preparedness Framework; on September 1, OpenAI confirmed that Astra is in fact the first model to cross that threshold, capable of finding and exploiting zero-day vulnerabilities without step-by-step human guidance, and said the model was not involved in the Hugging Face incident.^[7] The July 19 escalation that finally triggered detection involved agents running a different internal model from the same family as Astra, though the report notes it was a distinct model with different post-training.^[1]

A pause that ends when the next model ships is a schedule adjustment. This one has run longer than that, and it’s the first case we know of in which a frontier lab quarantined and rekeyed a model’s weights over the model’s own behavior. OpenAI frames the pause as temporary, tied to the safeguards it’s building for Astra.

AI Safety