Imagine discovering that a competitor spent six weeks running 28.8 million conversations with your flagship product. Not to use it, but to copy its brain.
That’s exactly what Anthropic alleges happened. Between April 22 and June 5, 2026, operators linked to Alibaba’s Qwen lab allegedly spun up roughly 25,000 fake accounts and systematically extracted Claude’s reasoning and coding capabilities. The goal: train Qwen on the stolen output.
No hacking. No breach. Just a competitor using Anthropic’s own API against it, at industrial scale.
The uncomfortable truth that nobody in the AI industry wants to talk about? This probably isn’t illegal.
The Anatomy of a Distillation Attack
Model distillation sounds technical, but the concept is deceptively simple. A smaller “student” model learns by imitating the outputs of a larger “teacher” model. When done legitimately, like Apple paying Google $1 billion annually for access to Gemini, it’s a standard technique for creating efficient, specialized models.
When done without permission, it’s what Anthropic calls an “adversarial distillation attack.”
Here’s how the Alibaba operation reportedly worked:
| Metric | Value |
|---|---|
| Fake accounts | ~25,000 |
| Total interactions | 28.8 million |
| Duration | April 22, June 5, 2026 |
| Queries per account | ~1,152 (avg) |
| Queries per account per day | ~26 (avg) |
The attackers kept individual account activity below standard rate limits. No single account behaved suspiciously. Spread across 25,000 accounts, the operation flew under the radar for six weeks.
Anthropic’s letter to US Senators Tim Scott and Elizabeth Warren described this as “the largest known distillation attack” in the company’s history, bigger than DeepSeek, Moonshot, and MiniMax combined.
The Technical Transfer
What exactly does distillation steal? When you query a frontier model like Claude, you’re not just getting an answer. You’re getting probability distributions, the model’s confidence-weighted predictions across its entire vocabulary. Claude might say “the next word is ‘however’ with 68% probability, ‘but’ with 21%…”
A student model trained on these distributions inherits far more than just correct answers. It learns the model’s reasoning patterns, its stylistic tendencies, and its decision-making heuristics. The student never sees the original training data, but it gets a compressed version of everything the teacher learned.
DeepSeek’s publicly released R1-Distill-Qwen-32B scored 94.3% on MATH-500, nearly matching much larger frontier models. The benchmark gap between original and student is startlingly small.
But benchmarks lie. What distillation can’t transfer is the long tail, the rare cases, the robust alignment, the obscure domain knowledge that exists in the original model’s training distribution. When you iterate across generations on distilled data, this loss accumulates. A 2024 Nature study by Shumailov and colleagues documented this “model collapse” phenomenon: trained purely on synthetic data, models deteriorate within roughly nine generations.
The remedy? Keeping just 10% real source data every generation. That 10% is exactly what distillation can’t steal.
The Hypocrisy That Haunts This Debate
Here’s where things get messy.
Anthropic built Claude by training on seven million books pirated from LibGen’s “shadow library”. They settled for $1.5 billion. They’ve been accused of scraping virtually the entire internet for training data. They bought physical books and destroyed them while scanning.
So when Anthropic runs to Congress crying theft, the response from developer communities is… predictable.
The sentiment on forums is sharp: when Anthropic was setting the precedent by pirating massive amounts of copyrighted material, it wasn’t a problem. Now the same thing is happening to them, and suddenly it’s a national security crisis.
The difference, some argue, is that Alibaba paid for its usage. Anthropic paid the people whose content they scraped exactly nothing until forced to by courts.
This isn’t just moral posturing. It cuts to the legal heart of the matter.
The Legal Catch-22
Anthropic’s entire legal argument rests on the claim that training AI models on data is “transformative” and thus falls under fair use. That’s how they justify scraping the internet without paying creators.
But if training on copyrighted data is transformative fair use, then Alibaba training on Claude’s outputs should be equally transformative. By Anthropic’s own argument, no wrongdoing has occurred.
Here’s the logical trap:
| If AI training is… | Then… |
|---|---|
| Fair use (Anthropic’s position) | Distillation attacks are perfectly legal. Claude’s outputs are just data to learn from. |
| Not fair use | Anthropic owes billions to every creator whose work trained Claude. The entire business model collapses. |
This is why Anthropic sent a letter to Congress rather than filing a lawsuit. There’s no clear legal theory that helps them here. They need new laws, not existing ones.
The legal reckoning is running in parallel: numerous copyright infringement lawsuits are pending against OpenAI, Anthropic, Meta, and others. No one has a clean answer to the question “Is learning from other people’s data, whether websites or model outputs, theft or progress?”
The Geopolitical Escalation
Anthropic is framing this as a national security issue. The company doesn’t offer Claude commercially in China at all, citing security reasons. The Chinese labs allegedly used commercial proxy services running “hydra cluster architectures” to route around regional blocks.
The US labs argue that distilled models hit the market without safety guardrails and undermine chip export controls, because the rapid progress of Chinese labs is misread as proof that those controls don’t work.
Anthropic is calling for penalties against distillation actors, tougher export controls, and easier information sharing among US firms. Legislative initiatives are already under review in the US Congress that would sanction Chinese companies for unlawfully accessing the outputs of US models.
The irony isn’t lost on industry observers. As one analysis noted, Qwen has captured 20% of OpenRouter traffic and benchmark scores that make Claude sweat. If distillation were the only mechanism behind Qwen’s performance, the explanation would be convenient for Anthropic, but many experts believe building a genuinely competitive model requires more than just copying outputs.
Apple Shows How It’s Done
Contrast the Alibaba allegations with Apple’s approach. At WWDC 2026, Apple revealed its third-generation Foundation Models, built openly on Google’s Gemini technology. The deal, signed in January 2026, reportedly costs Apple around $1 billion per year.
Apple distills with a license, a contract, and a price tag. Google gets paid. The cooperation is public. Both sides negotiated terms down to the assurance that no Apple user data flows to Google.
This model, distillation as a licensing business rather than a gray-zone arms race, is likely to become the industry’s reference case. It’s also why Microsoft CEO Satya Nadella took a swipe at the competition when unveiling MAI-Thinking-1, boasting it was trained “from scratch” on clean, commercially licensed data.
The Model Collapse Problem Nobody’s Solving
The deeper technical concern isn’t about who stole what. It’s about what happens when every model is trained on every other model’s outputs.
The Nature study measured this precisely. Perplexity degraded from 34 into the 50s across generations of pure synthetic training. After roughly nine generations, output degenerated into meaningless repetitive lists. Common cases reproduce abundantly while rare cases vanish.
The internet is already flooding with AI-generated content. New web pages increasingly contain AI-generated text. If the next generation of models trains on this data, the distribution narrows. Tails vanish. Diversity collapses.
This is the real existential threat to AI progress, not Chinese labs copying American models, but the recursive contamination of the training data ecosystem.
Chinese AI labs like Tencent are releasing models under Apache 2.0 licenses, demonstrating a different approach to open-source AI that challenges the US model of closed, proprietary systems.
Who Actually Owns the Knowledge Inside a Model?
This entire dispute raises a question the industry has been avoiding: who owns the patterns inside a neural network?
If I read a textbook and learn physics, the knowledge becomes mine. No one can “steal” my understanding of gravity by listening to me explain it. But if I pay for API access to Claude and use the responses to train a cheaper model, Anthropic calls it theft.
The difference is that Anthropic wants to sell the same knowledge repeatedly. They built a business model on monetizing access to their model’s capabilities. Distillation breaks that model by making the knowledge replicable.
But replicability is a feature of digital information, not a flaw. The entire internet economy is built on the fact that copying data costs nearly nothing. AI companies can’t simultaneously argue that fair use protects their training on the world’s data and that their own outputs should be treated as protected trade secrets.
What Happens Next
Congress is weighing legislation. Export controls are tightening. Anthropic is pushing for regulatory protection while simultaneously facing its own copyright reckoning.
The smart money says we’ll see something like the “safe harbor” framework that emerged for early internet platforms: you can’t be sued for what your users do, but you have to respond to takedown notices. Applied to distillation, this might mean API providers get legal cover if they implement “reasonable” detection measures.
But technical detection is an arms race. Rate limits, behavioral analysis, IP blocking, all can be evaded with enough accounts and proxies.
The real solution might be simpler: make distillation economically pointless. If frontier models are cheap enough to use directly, the incentive to train a cut-rate clone disappears. That’s why Microsoft is pushing its own models, why Apple is paying for access, and why the entire industry is racing toward commoditization.
The Bottom Line
Anthropic has a real problem. A competitor extracted its most valuable capabilities on an industrial scale. But the tools Anthropic wants Congress to deploy would also break the legal foundation the AI industry was built on.
The dildo of consequences, as one developer bluntly put it, rarely arrives lubed.
For now, the distillation wars continue. Alibaba’s stock barely dipped. Claude keeps answering questions. And somewhere, someone is training the next generation of models on the output of this very article.
