When the Reddit post dropped claiming Tencent had compressed Hy4-preview from a whopping 1.5TB down to roughly 200GB in GGUF format while retaining 98% of its original performance, the reaction was immediate and visceral. “WTF! That’s insane”, was the top comment, and honestly? That’s a fair response.
Here’s the math that’s making people’s heads spin: that’s an 87% size reduction with a mere 2% performance hit. In a world where we’ve been conditioned to accept significant quality degradation at much gentler compression ratios, this feels like someone broke the rules.
Let me be clear about what we’re looking at here. Hy4-preview is Tencent’s flagship open-weight Mixture-of-Experts model: 770 billion total parameters, 49 billion activated per token, and a one-million-token context window. It’s the kind of model that normally requires an eight-GPU node just to serve the FP8 variant. The fact that someone’s managed to squeeze it into something approaching consumer-hardware territory is genuinely remarkable.
But before we break out the champagne, let’s dig into what’s actually happening here, why it works, and whether 98% is as good as it sounds.
The 87% Shrink: What Actually Happened
The compression was done using Tencent’s own AngelSlim toolkit, which the company positions as “a more accessible, comprehensive, and efficient toolkit for large model compression.” The output is a GGUF-format quantized model, the same format that’s been powering local inference through llama.cpp for years.
The math breaks down like this:
| Metric | Original | Compressed | Reduction |
|---|---|---|---|
| Size | 1,500 GB | ~200 GB | 86.7% |
| Performance | 100% | ~98% | 2% loss |
| VRAM required | Multiple H100s | Still substantial, but tractable | , |
For context on how aggressive this quantization level is: we’re talking about what amounts to a sub-2-bit average per weight. Earlier attempts at this level of compression on models like Hy3 showed “slightly worse” outputs with “no syntax errors” but reduced forward-thinking capability. Hy4-preview apparently handles it far better.
Why This Compression Works So Well
Here’s the part that’s genuinely interesting from a technical standpoint. There are two factors at play that explain why Hy4-preview takes compression unusually well.
1. It’s Under-Trained (In a Good Way)
Tencent is explicit that this is a preview release with “headroom left in both pre-training and post-training.” Think of the model’s weights as an uncarved block of marble. A heavily trained model has been refined to its limits, every weight is precisely tuned, and shaving off precision destroys that fine-tuning. An under-trained model has redundancy built in. The weights are less precise, less “locked in”, so quantization error has less of an impact.
As one commenter on the original thread put it: “I think under-trained models are easier to quant. A model that is near capacity will lose more with compression compared to one that has plenty of room for more data.”
The more refined a model is, the more distributed its knowledge representation becomes across weights, and the more damage you do when you lop off precision. An early checkpoint has slack, the weights aren’t carrying maximal information density yet.
2. MoE Architecture Helps
Hy4-preview’s Mixture-of-Experts backbone plays a significant role. With 256 routed experts per layer (only 77 layers, with one dense first layer, but the routing logic is what matters), the model is already designed for sparse activation. When you quantize aggressively, the expert routing mechanism, which essentially decides which “brain regions” fire for a given token, is remarkably robust to noise.
This lines up with a pattern people have noticed in the quantization community: compression works well on large models but not so much on small models. The redundancy in massive parameter counts acts as a natural error-correction mechanism. 770 billion parameters gives the model a lot of room to absorb quantization error without compromising output quality.
What 98% Actually Means
Now for the uncomfortable question: what does “98% of original performance” actually claim?
The benchmark table Tencent published for the full-precision model is impressive on its own:
| Benchmark | Score |
|---|---|
| GPQA Diamond | 92.3 |
| Terminal-Bench 2.1 | 85.4 |
| MCP-Atlas | 83.7 |
| SWE-bench Multilingual | 82.9% |
| Toolathlon-Verified | 74.1 |
| SWE-Bench Pro (public) | 65.7 |
| DeepSWE | 64.3 |
| SkillsBench V1 | 62.9 |
| Humanity’s Last Exam (with tools) | 55.4 |
| APEX-Agents | 37.1 |
If the compressed version genuinely achieves 98% of these numbers, that’s world-altering. But the AI community’s immediate skepticism, “come on man we know it’s going to be benchmaxxed to hell”, has merit.
Here’s what I’d note: even if the benchmarks are cherry-picked or slightly inflated, the size reduction itself is verifiable. A 200GB GGUF file that runs and produces coherent outputs at near-full quality is a concrete, measurable achievement. The debate isn’t whether it’s small, it’s whether it’s good.
One tester of the earlier Hy3 Q1 compression by AngelSlim reported: “The outputs will be slightly worse but it is still very good. It won’t make syntax errors just won’t think ahead as far.” Extrapolating to Hy4-preview, the 2% figure could represent a genuine capability shift, fewer long-horizon reasoning chains, marginally less sophisticated planning, while still producing valid, useful responses.
What This Means for Local AI
The future of local models is brighter than expected. That’s not hype, it’s a logical consequence of what’s happening here.
Let’s put this in context. The original Hy4-preview model, at 1.5TB, requires an eight-GPU node. Even the FP8 variant published by Tencent needs substantial enterprise hardware to serve. The official serving recipes target a tensor-parallel size of eight.
At 200GB in GGUF format, the compressed model becomes:
- Offloadable to reasonable hardware, still needs substantial RAM/VRAM, but we’re talking about a single high-end workstation now, not a GPU cluster
- Streamable from NVMe, with the caveat that “NVMe is magnitudes slower than RAM unfortunately… so very slow tps if we do offload to NVMe”, as one community member pointed out
- Potentially the first H100-class model on consumer hardware
One Reddit user captured the sentiment perfectly with a touch of humor: “Awesome. Only 190 GB more to go and then I will run it on my RTX 3080.”
That’s hyperbolic, but the direction is right. We’re watching the trajectory from “requires a data center” to “requires a powerful desktop” to, eventually, “runs on a laptop.”
The Dark Side of the Model Size Obsession
Let’s take a step back, because there’s a bigger story here than Tencent’s latest flex.
The AI industry has spent the last few years in a parameter-count arms race. The shift from parameter-count obsession to architectural specialization was already happening before this announcement, but compression breakthroughs like this accelerate it dramatically.
Here’s the uncomfortable truth this reveals: much of the “intelligence” in these massive models is redundant, distributed, or simply not needed for most tasks.
Hy4-preview activates only 49 billion of its 770 billion parameters per token. That’s 6.4%. The compression to 200GB while retaining 98% performance suggests that even the stored weights carry significant redundancy.
This isn’t a new insight, Tencent’s own prior releases have shown smaller models punching above their weight class. And the Youtu-LLM-2B saga demonstrated that efficiency-focused models can outperform much larger competitors. But each new breakthrough makes the argument more compelling.
The Competitive Landscape Shifts
This compression achievement arrives at a pivotal moment in the open-weight wars.
The number of open-weight models has exploded this year: Qwen3-Coder-Next, Kimi K2 from Moonshot AI, MiniMax-M2, Devstral 2 from Mistral, and OpenAI’s own Gpt-Oss experiment. Tencent’s Hy4-preview joins that field with the largest parameter count of them all.
But what matters isn’t just that Tencent shipped a big model. It’s that Tencent is also shipping the compression tools to make that model practical. AngelSlim is open source. The GGUF conversion is community-reproducible. That combination, the model plus the tools to shrink it, is what changes the competitive equation.
Consider the pricing implications from the Hy4-preview API comparison:
| Per million tokens | Hy4 preview | GPT-5.6 Sol (base) | GPT-5.6 Sol (large context) |
|---|---|---|---|
| Input | $0.834 | $4.00 | $8.00 |
| Output | $2.501 | $20.00 | $30.00 |
| Cache read | $0.042 | $0.40 | $0.40 |
At roughly an eighth of the cost for output tokens, the expensive half of agentic workloads, and with self-hosting now feasible at dramatically reduced hardware requirements, the pressure on closed-model pricing is only going to intensify.
This is part of Tencent’s broader open-source ecosystem strategy, which has been deliberately aggressive about making frontier-adjacent AI accessible. The Qwen3-TTS landing in llama.cpp was another data point in the same trend: making once-enterprise-only AI runnable on consumer hardware.
And if you’re looking for evidence that this efficiency push is a strategic choice rather than an accident, consider Tencent’s WeDLM-8B experiment, a diffusion-based LLM architecture that abandons autoregressive generation entirely. Tencent isn’t just optimizing within existing paradigms, it’s questioning the assumptions underneath them.
Where We Go From Here
The 98% figure demands scrutiny. The claims need independent verification, proper benchmarks on the compressed checkpoint, and testing across diverse workloads. “Benchmaxxing” is a real risk.
But even with the skepticism priced in, the trajectory is clear. If 200GB GGUF is real and 98% is even remotely accurate, the next few iterations will make these numbers look conservative.
The implications ripple outward:
- Local-first AI becomes viable for frontier-class models. We’re talking about the difference between “can’t run it at all” and “runs on a beefy workstation.”
- The cost of AI deployment collapses. Self-hosting a compressed frontier model removes the per-token tax entirely.
- The model-size war becomes irrelevant for most use cases. As the industry has been discovering, a 200GB model at 98% of frontier quality beats a 1.5TB model you can’t afford to run.
The race for bigger models isn’t over. But it’s becoming clear that the race for smarter models, and smaller representations of them, is where the real wins are happening.
One commenter summed it up: “The future of local models is brighter than expected. We can still compress them further.” Tencent just proved that in the most dramatic way possible.
The question that remains: who’s going to be the first to get this running on a phone? Because historically, that’s where the community always ends up, and this particular model might actually justify the effort.
