Xiaomi quietly dropped MiMo-V2.5-DFlash on Hugging Face this week, and the local LLM community is doing what it does best: refreshing the page, checking the file tree, and asking the same question, will this work with llama.cpp?
The model card is empty. The README is a ghost. But the files are real: 311 billion parameters, FP8 weights, and a dedicated dflash directory that has the speculative decoding crowd buzzing. This isn’t just another model upload. It’s a potential step-change in what’s possible on consumer hardware.
The Quietest Big Model Launch of the Year
Xiaomi’s MiMo family has been quietly building a reputation. The Xiaomi’s earlier MiMo-V2-Flash 309B model with DFlash already demonstrated that the company could punch above its weight class. But MiMo-V2.5-DFlash is different. It’s not just a bigger model, it’s a smarter architecture.
The Hugging Face repo is spartan. The model card is empty. The README is a ghost town. But the files tell the story: 311 billion parameters, FP8 quantization, and a dedicated dflash directory that has the speculative decoding community refreshing their browsers. The model sits at 311B params with F32, BF16, and F8_E4M3 tensor types available. It’s licensed MIT, which means the open-source community can run wild with it.
What Is DFlash, and Why Should You Care?
Speculative decoding is the art of using a small, fast model to draft tokens that a large model verifies in parallel. The bottleneck has always been the drafter. EAGLE-3, the previous state of the art, drafts autoregressively, one token at a time. That caps practical speedups around 2-3× because the drafter itself is sequential.
DFlash throws that constraint out the window. Developed by Z Lab and published at ICML 2026, it uses a lightweight block diffusion model to draft an entire block of tokens in a single parallel forward pass. The results are striking: up to 6× lossless acceleration on Qwen3-8B, nearly 2.5× faster than EAGLE-3.
The key insight is deceptively simple. Large autoregressive LLMs’ hidden features already contain information about multiple future tokens. DFlash doesn’t ask a tiny diffusion model to predict the future from scratch, it conditions the draft model on context features extracted from the target model itself.
The architecture is elegant: after prefill or verification, hidden features are extracted from layers uniformly sampled across the target model, fused through a lightweight projection, and injected directly into the Key/Value projections of every draft model layer. This is the crucial difference from EAGLE-3, which feeds target features only as input to the first layer. In DFlash, every layer gets the full context, so acceptance length scales with depth.
Why Diffusion Drafting Changes the Game
The bottleneck in speculative decoding has always been the drafter. EAGLE-3, the previous state of the art, drafts autoregressively, one token at a time. That means drafting cost grows linearly with the number of tokens. To keep latency low, EAGLE-3 is forced to use an extremely shallow architecture (a single transformer layer), which severely limits draft quality.
DFlash breaks this trade-off. Because diffusion drafters generate all tokens in a single parallel forward pass, drafting cost is essentially flat regardless of how many tokens you produce. A multi-layer DFlash generating 16 tokens has lower latency than a 1-layer EAGLE-3 generating 8 tokens. Deeper model, more tokens, less time.
The numbers from the Z Lab paper are striking. On Qwen3-8B with greedy decoding, DFlash achieves a 5.20× speedup on GSM8K versus 2.13× for EAGLE-3. On MATH-500, it’s 6.17× versus 2.18×. On AIME24, 5.91× versus 2.25×. The pattern is consistent: DFlash delivers over 2.5× higher speedup than EAGLE-3 on most tasks.
| Greedy (temp=0) | GSM8K | Math500 | AIME24 | AIME25 |
|---|---|---|---|---|
| Acceptance length τ | 3.38 | 4.61 | 4.12 | 4.07 |
| Speedup | 2.83× | 3.73× | 3.43× | 3.35× |
| Sampling (temp=1) | GSM8K | Math500 | AIME24 | AIME25 |
|---|---|---|---|---|
| Acceptance length τ | 3.29 | 4.12 | 3.23 | 3.24 |
| Speedup | 2.76× | 3.31× | 2.66× | 2.65× |
The Architecture That Makes It Work
The DFlash pipeline is elegant in its simplicity. After prefill or verification, hidden features are extracted from layers uniformly sampled across the target model. These features are fused through a lightweight projection and injected directly into the Key/Value projections of every draft model layer, stored in the KV cache.
This is the architectural insight that makes DFlash different. EAGLE-3 feeds target features only as input to the first layer, meaning the signal dilutes as you add layers. DFlash gives every layer the full context, so acceptance length scales with depth. The draft model reuses the embedding and LM head from the target model, only the few intermediate layers are trained, keeping the parameter count minimal.
What the Community Is Seeing
The early reports from the NVIDIA Developer Forums are encouraging. One user running MiMo-V2.5 with DFlash on a 2× DGX Spark pair reported speeds jumping from 22 to 67 tok/s depending on workload. That’s a 3× improvement on hardware that costs less than a single high-end GPU.
Another user running the model on 2×24GB cards with VRAM offload to 96/128GB DDR5 reported baseline speeds of 8-10 tk/s. With DFlash, the expectation is that speed could double, making a 300B+ model genuinely usable on consumer-grade hardware.
The community is particularly excited about the separate MTP model that was also shared. The reason llama.cpp doesn’t work with the MTP head yet is that it has trouble identifying the MTP layers. A separate MTP model might work instead, and the DFlash weights are already available for anyone willing to GGUF them.
How DFlash Compares to the Competition
The numbers don’t lie. On Qwen3-8B, DFlash delivers a 5.20× speedup on GSM8K versus 2.13× for EAGLE-3. On MATH-500, it’s 6.17× versus 2.18×. On AIME24, 5.91× versus 2.25×. The pattern is consistent across math, code, and chat benchmarks.
But the real story is what happens under sampling. At temperature=1, DFlash still delivers roughly 4.5× acceleration for reasoning models. The acceptance lengths remain strong: 3.29 on GSM8K, 4.12 on MATH-500, 3.23 on AIME24. This isn’t a greedy-only trick, it works in the real world where you want diversity in generation.
The Hardware Reality Check
Let’s be honest about what this means for local inference. The model is 311B parameters. Even with FP8 quantization, you’re looking at roughly 311GB of VRAM for the full model. That’s not fitting on a single consumer GPU.
But the community is already finding workarounds. On 2×24GB cards with VRAM offload to 96/128GB DDR5, users are getting 8-10 tk/s baseline. With DFlash, that could jump to 16-20 tk/s. On a 2× DGX Spark pair, users are reporting 22→67 tok/s depending on workload.
The Xiaomi’s MiMo-V2.5-Pro-UltraSpeed 1T model speed benchmarks show what’s possible when you combine DFlash with aggressive quantization and optimized hardware. The Pro-UltraSpeed variant hits 1000 tokens per second on a 1T parameter model, but that requires FP4 quantization and TileRT system-level optimizations that aren’t available to local users yet.
The DFlash vs. MTP Confusion
There’s been some confusion in the community about how DFlash relates to Multi-Token Prediction (MTP). They’re related but not the same. MTP is a training technique where the model learns to predict multiple future tokens simultaneously. DFlash is a speculative decoding method that uses a diffusion model as the drafter.
The key difference: MTP still drafts autoregressively. DFlash drafts in parallel. That’s why DFlash can achieve higher speedups, it’s not limited by the sequential nature of autoregressive generation.
One user on the NVIDIA forums clarified: “DSpark is known for reducing token prices, not DFlash. DFlash is known best for improving single user Qwen 3.6 27B inference better than MTP.” This is an important distinction. DFlash is optimized for single-user scenarios where latency matters most, not for batch processing where throughput is king.
What This Means for Local LLM Enthusiasts
The 300B parameter class has been a frustrating middle ground. Models like Xiaomi’s earlier MiMo-V2-Flash 309B model with DFlash showed promise, but the inference speeds on consumer hardware were painful. DFlash changes the equation.
If you’re running on 2×24GB cards with system RAM offload, you’re looking at 8-10 tk/s baseline. With DFlash, that could hit 16-20 tk/s. That’s the difference between “painful to use” and “actually usable for real work.”
The 80-160B model gap and unified memory limitations have been a persistent pain point for local LLM enthusiasts. Models in that range don’t fit on consumer GPUs but are too small to justify enterprise hardware. DFlash doesn’t solve the memory problem, but it makes the speed problem much more manageable.
The Benchmark Picture
On BenchLM’s provisional leaderboard, MiMo-V2.5 scores 65/100 against DeepSeek V4 Flash’s 57/100. The sharpest advantage is in agentic tasks, where MiMo-V2.5 averages 65.8 against 49.1. The single biggest benchmark swing is Terminal-Bench 2.0: 49.1% to 65.8%.
But DeepSeek V4 Flash hits back in coding, averaging 57.1 against 56.1. The choice depends on your workload. If you’re building agentic systems, MiMo-V2.5 is the clear winner. If coding is your priority, DeepSeek V4 Flash has a slight edge.
The Xiaomi’s MiMo-V2.5-Pro open weights model capabilities show that the Pro variant is even more capable, rivaling Claude Opus 4.6 in demanding agentic workloads. But the DFlash variant is the one that makes local inference practical.
What’s Next for DFlash and llama.cpp
The big question is when DFlash will be wired into llama.cpp. The community is actively working on it, and the separate MTP model release suggests Xiaomi is thinking about compatibility. The main blocker is that llama.cpp has trouble identifying the MTP layers, but a separate MTP model might work.
For now, the best path to running MiMo-V2.5-DFlash locally is through SGLang, which already supports DFlash. The setup is straightforward:
pip install "git+https://github.com/sgl-project/sglang.git@refs/pull/16818/head#subdirectory=python"
python -m sglang.launch_server \
--model-path XiaomiMiMo/MiMo-V2.5-DFlash \
--speculative-algorithm DFLASH \
--speculative-draft-model-path XiaomiMiMo/MiMo-V2.5-DFlash \
--tp-size 1 \
--dtype bfloat16 \
--attention-backend fa3 \
--mem-fraction-static 0.75 \
--trust-remote-code
For those with more modest hardware, the Luce DFlash’s 2x throughput technical explanation provides a deeper dive into how the technique works at the hardware level.
The Bottom Line
Xiaomi’s quiet upload of MiMo-V2.5-DFlash is more than just another model release. It’s a signal that speculative decoding is moving from academic papers to production-ready implementations. The DFlash technique, developed by Z Lab and published at ICML 2026, represents a genuine breakthrough in inference acceleration.
The model itself is competitive. On BenchLM’s provisional leaderboard, it scores 65/100 against DeepSeek V4 Flash’s 57/100. Its agentic performance is particularly strong, with a 65.8% score on Terminal-Bench 2.0 versus 49.1% for DeepSeek V4 Flash.
But the real story is what DFlash enables. A 300B+ parameter model running at usable speeds on consumer hardware is no longer a pipe dream. It’s on Hugging Face, it’s MIT licensed, and the community is already figuring out how to make it work.
The DeepSeek DSpark speculative decoding breakthrough showed what’s possible with optimized speculative decoding. DFlash takes that further, proving that diffusion models don’t need to compete with autoregressive LLMs in generation quality, they just need to be great drafters.
The Bottom Line
Xiaomi’s MiMo-V2.5-DFlash is a genuine breakthrough for local LLM inference. The combination of a 311B parameter model with DFlash’s block diffusion speculative decoding could double your inference speed on hardware you already own.
The model is on Hugging Face, it’s MIT licensed, and the community is actively working on llama.cpp compatibility. If you’ve been sitting on the fence about running large models locally, this might be the push you need.
The Xiaomi’s MiMo V2.5 1T model achieving 3000 tokens per second shows where this technology is heading. For now, the DFlash variant gives us a taste of what’s possible on consumer hardware. And it’s delicious.
