An AI agent broke into Hugging Face's network — and when the team went to investigate, commercial models mistook them for the attacker
- Hugging Face published a disclosure on July 16 saying that earlier that week it found part of its production infrastructure — the internal systems that actually serve users — had been compromised. The entire operation was run end to end by an automated AI agent system.
- The way in was a malicious dataset. It exploited two flaws in the dataset-processing path to execute code on the machine handling it, then escalated to full control of that machine, harvested cloud and cluster credentials, and spent a weekend moving laterally into several internal clusters.
- What got touched: some internal datasets, plus credentials used by several services. No evidence of tampering was found in public models, datasets, or Spaces — the small live apps users host on the platform. Container images and published packages verified clean. Whether partner and customer data was affected is still under review.
- For forensics, the team first reached for the strongest models available through commercial APIs. The requests were blocked by safety guardrails, because what they needed to submit was exactly the attacker's real commands and the exploit code used to get in. Switching to open-weight GLM 5.2 running on their own machines is what let them work through all 17,000-plus events.
- Hugging Face's advice to defenders: pick a model you can run in your own environment and validate it in advance. Don't go shopping for one on the day of an incident.
What makes this intrusion different
On July 16, Hugging Face published a security incident disclosure: earlier that week it detected and contained a breach of part of its production infrastructure — the internal systems that actually serve users.
Over a single weekend, the attacker went from code execution to lateral movement across multiple internal clusters, leaving more than 17,000 logged actions. Those tens of thousands of actions ran across a swarm of short-lived machines, and the command channel relocated itself. On the defense side, AI compressed a log reconstruction that would have taken days into a few hours.
The disclosure splits findings into two columns: what's been established, and what hasn't.
- Unauthorized access to some internal datasets
- Access to credentials used by several services
- No evidence of tampering in public models, datasets, or Spaces
- Container images and published packages verified clean
- Whether partner and customer data was affected — assessment ongoing, and anyone impacted will be contacted directly
- Which model powered the attacker's agent remains unknown
The attack arrived as a dataset
It started at a weak point specific to AI platforms: the data-processing pipeline. A malicious dataset exploited two code-execution paths in dataset handling to run its own code on the processing machine. Everything after that was textbook internal network intrusion.
The dataset itself is ordinary input the platform handles every day, submitted through the normal channel.
Two code-execution paths in dataset handling let the attacker's own code execute on that machine.
From the privileges of a single processing job up to control of the host.
Cloud service and cluster credentials stored on the machine. These are the keys to everything else.
With the keys in hand, the attacker hopped machine to machine and reached several internal clusters — a cluster being a batch of machines grouped together to run one kind of workload.
The two entry points work on different principles, and they're clearer side by side.
Loading a dataset also executes the Python script its author shipped with it.
The author writes the script, the platform performs the load — and someone else's code ends up running on your machine.
A field in the dataset config file meant to hold data holds something else instead. The classic template-injection move is to get that content executed as a command when the system reads the config.
Like filling out a form and writing something in the "name" box that the system then runs. The disclosure says only "template injection in config" and gives no further mechanism.
One contrast here is worth pausing on. In the datasets library Hugging Face ships to everyone, version 4.0.0 from July 2025 removed script-based datasets entirely and retired the trust_remote_code flag along with them; the official migration path is converting to plain data formats like Parquet. So ordinary users on a current version lost the "loading a dataset also runs a script" path a year ago — yet in the platform's own backend processing pipeline, this class of code-execution path was still available to be used. Both paths used for initial access have since been closed.
To be precise about what the disclosure does not say: it doesn't state whether the backend pipeline runs that same datasets library, or whether the exploited path is the one 4.0.0 sealed. The comparison above places two publicly known facts next to each other; it isn't a conclusion the disclosure draws.
What broke in was an autonomous agent framework
Here's the shape of it. A program that advances step by step on its own, apparently built by modifying an off-the-shelf automated penetration testing tool — the kind meant for red teams and vulnerability research. Its tens of thousands of actions didn't run on one machine; they were spread across a swarm of short-lived machines, each destroyed after a handful of steps. The channel back to whoever was operating it hid inside public services and moved itself around. Which model it used is still unknown.
About that "off-the-shelf tool" layer — the industry calls it a harness, the scaffolding wrapped around a model: it feeds the model material, controls what the model can see, parses what comes back, and drives the loop forward. The model is the engine; the harness is the chassis and steering. Put the same engine in different chassis and the results diverge sharply.
Top: tens of thousands of actions executed across a fleet of short-lived machines. Bottom: the command channel hides inside normal traffic on public services and migrates to the next one. Drawn from the descriptions in the Hugging Face disclosure.
Both design choices serve the same goal: denying you a single clean cut. Throwaway machines are disposable by design, so blocking one buys nothing. A channel blended into normal traffic that also relocates makes blocklisting fixed addresses useless.
There's a second-order effect on forensics. When machines self-destruct, disk images and memory snapshots go with them, and what's left to reconstruct from is mostly the action log. That makes those 17,000-plus events the most complete record of the entire attack. (That connection is this site's inference; the disclosure doesn't state it.)
What the last publicly documented AI-driven attack looked like
This is precisely the "attacker that advances on its own" the industry has been warning about. The previous case documented in detail was GTG-1002, disclosed by Anthropic's threat intelligence team and detected in mid-September 2025. The two differ in how much was automated. The table below draws on outside material.
| Dimension | GTG-1002 (Sept 2025) | This incident (July 2026) |
|---|---|---|
| Targets | Roughly 30 entities across tech companies, financial institutions, chemical manufacturers, and government agencies | Hugging Face alone, driven deep into multiple internal clusters |
| Automation | AI handled 80–90% of tactical actions; humans signed off at checkpoints like moving from recon to attack and approving data exfiltration | Described in the disclosure as end-to-end, driven by an automated agent system |
| Model used | A jailbroken Claude Code | Still unknown |
How Hugging Face pieced the intrusion together
AI did two jobs on the defense side: one for detection, one for reconstruction.
On detection, the team runs an AI on watch duty that reads the operational records the systems emit and pulls real signals out of everyday noise. It was several signals correlating that finally surfaced this intrusion.
An analysis agent built a timeline out of 17,000-plus events
To work out what tens of thousands of automated actions actually did, Hugging Face pointed an analysis agent at the full attacker action log — more than 17,000 events. One pass produced four things at once.
That last item is where the workload really sat. The attacker mixed in decoy actions, and separating real impact from deliberate noise is the single most labor-intensive part of the job for humans. Planting decoys also costs the two sides very differently: every fake action a human attacker adds costs that human time, while an agent running on its own can add however many at a marginal cost near zero. (The cost argument is this site's inference; the disclosure notes only that decoy actions existed.)
With AI doing the pass, work that would have taken days finished in hours — fast enough to keep pace with the adversary.
Why commercial models refused the forensic work
What actually stalled Hugging Face in this incident was the models it pays for. In the disclosure's own words, the constraint on which models could be used for the analysis was one the team "did not anticipate."
At the start of the log analysis, Hugging Face reached for the strongest models available through commercial APIs. It didn't work. This kind of analysis requires feeding the model large volumes of real attack commands, the exploit code used to get in, and artifacts left by the command channel. Safety guardrails block that material on sight — and they can't distinguish an incident response engineer from the attacker.
Forensics and attack feed a model material that looks identical. The guardrail only sees what you submitted; it can't see why you submitted it.
So at the very start of the log analysis, exactly when tens of thousands of events most needed to be sorted out fast, the defenders were locked out by the tools they pay for. Whether the agent on the other side ran a jailbroken hosted model or an unrestricted open-weight one is still unknown; the only certainty is that it was bound by no usage policy at all. The side playing by the rules got stopped by the rules. The side ignoring them sailed through.
The same wall stopped only one side. The attacker was bound by no usage policy; the defender's forensic request was left at the door. Drawn from the descriptions in the Hugging Face disclosure.
Moving the model onto their own infrastructure routed around the wall
Hugging Face ultimately shifted the forensic analysis to GLM 5.2, running inside its own infrastructure. It's an open-weight model from Zhipu AI (Z.ai), released under the MIT license, which means you can download it and deploy it yourself — nobody is in a position to approve or deny what material you submit. The training data and full pipeline aren't public, so it's open weights, not open source.
Running on your own hardware buys a second thing beyond dodging the guardrail, and Hugging Face called it out separately: the attacker data and the credentials referenced in it never left the company's network. During incident forensics, shipping credential-bearing logs to an external API is itself a fresh exposure.
This isn't an argument against safety measures on hosted models, and we're sharing this feedback with the providers involved. Hugging Face security incident disclosure
Does switching to open weights mean settling for weaker security analysis
There's an independent benchmark worth consulting. Semgrep tested a range of configurations against the same IDOR access-control vulnerability dataset with the same prompt. GLM 5.2, given nothing but a prompt and no purpose-built scaffolding, scored 39% F1 — that's precision and recall combined into one number: how much of what it found was real, and how much of what's real it found, out of 100%, higher is better. It beat Claude Code running inside its own agent scaffolding.
Which leaves defenders with one piece of advice: before an incident happens, choose and validate a model you can run on your own infrastructure. It keeps guardrails from blocking you, and it keeps attacker data and credentials from leaking out.
What Hugging Face fixed, and what users should do
Five things were done afterward.
- Patched the root-cause flaws: both dataset code-execution paths used for initial access are now closed.
- Evicted the attacker's footholds in affected clusters and rebuilt the compromised machines.
- Revoked and rotated affected credentials and tokens, plus launched a broader precautionary key rotation.
- Added extra protections and stricter admission controls on the clusters.
- Improved detection and alerting so high-severity signals wake someone on call within minutes — weekends included.
The company also brought in an external cybersecurity forensics team to investigate and review its security policies and processes, and has reported the incident to law enforcement.
Here's the action list for users and defenders. The first three are for regular users, the last two for enterprise security teams.
A few conclusions to close on. AI attack tools that advance on their own are no longer hypothetical. Mounting a broad, sustained, multi-stage campaign used to be expensive; that cost has now dropped, and the whole thing runs at machine speed. For any online platform, user-uploaded datasets and model files — long treated primarily as content awaiting processing — now have to be defended against as potentially poisoned input. And on the defensive side, humans reading logs can no longer keep pace with machines, which means putting AI to work there too.
Hacking went from one person up all night at a keyboard to an AI program running the whole break-in
Hugging Face's internal network was breached — and when it went to investigate, the commercial models it pays for refused to help. The whole story on one illustrated page.
↓ One page · includes an animated diagram
Hugging Face is where the world's AI models and datasets collect — millions of people uploading and downloading. One thing it does every single day is process dataset files uploaded by strangers, and that's exactly where it got broken into.
✘ What it didn't: a dataset running its own smuggled code while being processed
Loading a dataset also executes the Python script its author attached, and a config field meant to hold data can hold a line that gets run as a command (template injection). The platform treated user-uploaded files as "content to process," not as "potentially poisoned input."
On July 16, Hugging Face disclosed that earlier that week, part of its internal production systems had been compromised. Work like this used to mean a person at a keyboard typing each step. This time nobody touched anything — an AI program ran the whole break-in itself.
One machine, long residency
Fixed command address
Hundreds of actions, tiring
Throwaway machines, destroyed after use
Channel hides in public services, relocates
17,000+ actions, never tires
Five steps: upload a poisoned dataset → run code on the machine processing it → escalate to the whole machine → harvest cloud and cluster credentials (the keys to every other machine) → use those keys to move laterally into several internal clusters. Steps three through five all happened over one weekend.
Confirmed as touched: some internal datasets and several service credentials. No evidence of tampering was found in public models, datasets, or Spaces. The real trouble came next, when Hugging Face went to work through those 17,000-plus log entries.
Working out what the attacker did means feeding a model the real commands it typed and the exploit code it came in with, in full. Commercial safety guardrails block that material on sight — they can't tell whether the person submitting it is an engineer fighting the fire or the attacker.
The side playing by the rules got stopped by the rules; the side ignoring them sailed through. Hugging Face ended up running the model in its own data center to finish reading the logs — with a bonus: the attack data and the credentials it referenced never left the company network.
More than 17,000 logged actions take a human days to work through one by one; an AI pass takes hours. And this isn't simple reading: the attacker mixed in fake actions, and separating real impact from deliberate noise is the most labor-intensive part of the job.
The disclosure says only "days compressed into hours" and gives no exact figures. This estimates 3 days (72 hours) against 6 hours, roughly 12×, with the bars drawn to that ratio. The order of magnitude is Hugging Face's own assessment and can't be verified externally.
Switching to an open model isn't settling, either. In Semgrep's July 2026 access-control vulnerability benchmark, GLM 5.2 scored 39 with nothing but a prompt, beating Claude Code running in its own scaffolding (32). That said, it's a single task and a single run, and not data from this incident.
uploading,
downloading.
origin, every day.
the keyboard.
across throwaway
machines, gone
after use.
public services —
and relocates itself.
all of it has to
go in:
- × Real attack commands
- × The exploit code
- × Channel artifacts
has no wall.
datasets, Spaces
found to be
tampered with
some internal datasets
and service credentials.
the rules got stopped
by the rules.