Here’s a sentence that should make every AI safety researcher and enterprise CISO sit up straighter: an open-weight, MIT-licensed model just beat OpenAI’s flagship cyber model on a multi-step offensive security task. And it did it at a fraction of the cost.
Vals AI dropped the news on X this week: DeepSeek V4.1 Flash 1017 outperformed GPT-6 Sol on their simulated corporate network penetration benchmark. GPT-6 Sol wasn’t even publicly available, it was tested through OpenAI’s Daybreak trusted access program, the gated channel for vetted security professionals. DeepSeek’s model does this for anyone with an API key and a few cents.
The implication isn’t subtle. As Vals AI put it: “The offensive capability previously limited to a closed-source, trusted access model is now available to anyone via the latest DeepSeek model.”
What the Benchmark Actually Shows
Before we spiral into doomscrolling, let’s look at what Vals AI actually did. They built a cybersecurity benchmark based on simulated corporate networks, not the toy problems that populate most public benchmarks. Models were given access to scenario infrastructure, an objective, and instructions to make as much progress as possible. Think: reconnaissance, lateral movement, privilege escalation, the full offensive playbook.
Two details matter here:
- They used the Pi agent harness, not DeepSeek’s own Harness. This matters because it removes the “vendor optimizing for their own scaffold” skepticism. The performance is coming from the model, not from a tuned agent wrapper.
- GPT-6 Sol went through Daybreak, OpenAI’s trusted access program that requires FIDO2 hardware keys, identity verification, and organizational approval. This is the version with reduced refusals for cybersecurity work, the one OpenAI deems safe enough for defensive practitioners.
And DeepSeek’s cheap, open, MIT-licensed model still won.
The 1017 Update Is a Bigger Deal Than It Looks
The “1017” suffix screams minor update, a dated refresh, same architecture, nothing to see here. The numbers say otherwise.
On Terminal-Bench 4.0, the benchmark for agentic command-line task completion, the 1017 update jumped from 31.2 to 42.3. That’s an 11.1-point gain that moved DeepSeek from trailing GLM-5.3 and GPT-5.6 Sol to leading both. On ProgramBench, the jump was even more dramatic in relative terms: 20.3 to 29.1, roughly a 43% improvement.
| Benchmark | Flash 1017 | Previous Flash | Kimi K3 | GLM-5.3 | GPT-5.6 Sol | Opus 5 |
|---|---|---|---|---|---|---|
| Terminal-Bench 4.0 | 42.3 | 31.2 | 12.6 | 37.9 | 39.9 | 51.8 |
| DeepSWE v1.1 | 77.0 | 74.2 | 67.5 | 66.9 | 73.0 | 74.0 |
| ProgramBench | 29.1 | 20.3 | 17.5 | 19.0 | 23.0 | 37.0 |
| Automation-Bench | 60.5 | 54.8 | 46.7 | 48.8 | 45.8 | 50.3 |
| Apex Bench v1.1 | 72.1 | 64.6 | 47.8 | 56.6 | 54.3 | 65.8 |
DeepSeek now leads all named competitors, including Opus 5, on four of the most agentic benchmarks: DeepSWE, Automation-Bench, Agents’ Last Exam, and Apex Bench. The places it still trails, like HLE and ProgramBench, are the pure reasoning tests where massive frontier models like Opus 5 retain an edge.
This is the pattern that matters: DeepSeek is winning where agents need to act, not just think.
The Economics of Offensive AI
Here’s where this gets uncomfortable for OpenAI’s business model.
GPT-6 Sol’s pricing sits at $2/$10 per million tokens for input/output. DeepSeek V4.1 Flash 1017: $0.15/$0.60 off-peak, $0.30/$1.20 peak. The gap is even more lopsided when you factor in DeepSeek’s cache-hit pricing at $0.003 per million tokens.
For a workload of one million uncached input and one million output tokens, DeepSeek costs $0.75 off-peak. Sol and Sonnet 5.5 run $12. Opus 5.5: $24. Astra: $60. That’s a 16x to 80x cost difference, depending on who you’re comparing against.
The standard rebuttal is that OpenAI’s models produce shorter outputs, so the effective cost per task is closer. Vals AI acknowledges this: DeepSeek spends significantly more tokens per task. But when the raw per-token price is 16x cheaper, you can burn through a lot more tokens and still come out ahead.
The third-party pilot from Kingy confirms this pattern. On seven hosted coding tasks, DeepSeek V4.1 Flash 1017 passed 14/14 first submissions at a reported cost of $0.015731, versus $0.057710 for GPT-6.1 Sol. Same acceptance rate, roughly 3.7x cheaper.
The Safety Question Nobody Wants to Answer
OpenAI rates GPT-6 Sol’s cybersecurity capability as “High” under its Preparedness Framework, below Critical, but still capable enough to require a trusted access program, hardware key verification, and identity checks. The entire Daybreak infrastructure exists because OpenAI believes GPT-6 Sol’s offensive capabilities are dangerous enough to gate behind explicit authorization.
DeepSeek just matched or exceeded that capability. MIT license. No safeguards. No access controls. Anyone can download it.
The U.S. government restrictions on DeepSeek in federal agencies suddenly look less like bureaucratic caution and more like recognition of an uncomfortable reality: the offensive capability frontier has an open-source component that no export control can fully contain.
OpenAI’s own numbers tell the story. On their static cyber-jailbreak test, GPT-6 Sol scores 79.8% defender success, up from 59.0% for GPT-5.6 Sol. They invested over 700,000 A100-equivalent GPU hours into automated red-teaming to find and patch jailbreaks. And a model with no jailbreak resistance whatsoever just outperformed the locked-down version on the task.
The safety framing flips here. If DeepSeek’s model is genuinely better at offensive security tasks, then OpenAI’s Daybreak access program isn’t containing a capability gap, it’s creating a legal moat around a capability that’s already escaped. That’s arguably worse for security, because it gives approved defenders a false sense that they have access to something exclusive.
What DeepSeek Did Differently
The technical report title says a lot: “DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression.”
This isn’t just efficiency theater. Offensive security tasks involve long agent transcripts, tool call histories, and multi-turn interactions that accumulate context aggressively. The KV cache is the bottleneck for exactly these workloads. DeepSeek’s compression approach directly targets the memory overhead that makes long-context agentic tasks expensive and slow.
The model also ships without a standard Jinja chat template, instead providing a Python reference encoder and a Rust-based toolkit called deepseek-recipe for production prompt formatting. It’s a friction point for integration, but it’s not a capability limitation.
For anyone who’s followed DeepSeek’s trajectory of efficiency breakthroughs, this is a continuation of the same playbook: aggressive system-level optimization, open weights, and pricing that rewrites the cost-performance curve.
The “GPT-5.6 Sol Beats GPT-6 Sol” Irony
Here’s the funniest subplot in this whole saga. Multiple developers working with these models have reported that GPT-5.6 Sol often outperforms GPT-6 Sol in practice. The consensus from developer forums is that the older model’s less aggressive safety posture sometimes produces better results on agentic tasks, particularly when the task involves anything security-adjacent.
One developer put it bluntly: “I switched back.” The numbers from OpenAI’s own jailbreak tracker support this indirectly, GPT-5.6 Sol’s weaker safeguards (59.0% defender success) are exactly why the UK AI Security Institute broke it in hours back in July. The model is more capable on offensive tasks precisely because it’s less constrained.
This suggests something uncomfortable about the frontier AI safety paradigm. If OpenAI’s newest models are being held back by their own safeguards to the point where an open-source competitor with zero safety infrastructure outperforms them on the gated tasks, then the “trusted access” model isn’t just leaking capability, it’s actively making the approved tools worse than the unapproved alternatives.
What This Means for Defenders
For security teams, the practical takeaways are immediate:
- Test DeepSeek V4.1 Flash 1017 before assuming you need frontier pricing for every security workload. The cost-performance advantage is too large to ignore, and the benchmark evidence now covers offensive security specifically.
- Don’t assume Daybreak access gives you exclusive capability. If Vals AI’s benchmark holds up under broader evaluation, the gap between “trusted access” models and open weights is closing, possibly faster than either OpenAI or its enterprise customers expected.
- Run your own paired evaluations. The DeepSeek pricing efficiency only matters if the model solves your actual tasks. Freeze your scenarios, your acceptance criteria, and measure cost per accepted result, not just benchmark scores.
- Watch the regulatory landscape. The open-source shift that started with coding benchmarks has now reached offensive security. If China’s proposed export controls on AI technologies pass, this entire conversation changes.
The Uncomfortable Bottom Line
The frontier AI race has a new dynamic: capability keeps escaping containment, and it’s doing so at ever-lower prices.
OpenAI built Daybreak as a controlled channel for high-risk cyber capabilities. DeepSeek built an MIT-licensed model that matches those capabilities without needing anyone’s permission. The gap between “safety infrastructure” and “real-world capability” isn’t closing, it’s being erased from the bottom up.
Vals AI plans to release the full evaluation in the coming weeks. The numbers will get scrutiny, the harness will get criticized, and the usual benchmark caveats will apply. But the direction is clear. The offensive capability frontier is no longer gated behind trusted access programs and hardware keys. It’s on Hugging Face, under an MIT license, priced at less than a dollar per million tokens.
The question for OpenAI isn’t whether DeepSeek’s model is actually better, it’s whether the entire Daybreak architecture makes sense when the capability it’s designed to gate is available for $0.75 off-peak from a model that doesn’t refuse anything.
Spoiler: it doesn’t. And that’s a conversation no one in the AI safety community is ready to have.
