Here’s a number that should make every Mac user who’s ever waited on Ollama sit up: 144 tokens per second on a Qwen3.8-27B model, running locally on an M5 Max MacBook Pro. That’s not a typo, and it’s not a cherry-picked benchmark from a vendor with something to hide. It’s the headline figure from Inco AI’s new open-source inference engine, Splash, and the numbers are remarkably transparent for a launch this aggressive.
The company’s official launch post lays out the full methodology: same 48 GB M5 Pro, same NVIDIA SPEED-Bench coding prompts, reasoning mode enabled, HTTP serving throughout. No tricks. And the results are genuinely impressive, on Qwen3.8-27B, Splash decodes at 74 tok/s on short prompts, which is 2× the speed of the next-fastest engine they tested (oMLX at 38 tok/s). At 32K context, it’s still 54 tok/s while Ollama drops to 19.
But here’s where things get spicy: the same specialization that makes Splash this fast is also its hardest ceiling. This is the first inference engine I’ve seen that’s built around the model rather than around a model zoo, and the engineering tradeoffs are fascinating.
The “Built Around the Model” Philosophy
Most inference engines, Ollama, llama.cpp, MLX, are designed to run anything you throw at them. Maximum flexibility, maximum compatibility, decent performance across the board. Inco’s approach inverts this completely.
Splash supports exactly two models at launch: Qwen3.8-27B (dense, 17.4 GB package) and Qwen3.6-35B-A3B (a mixture-of-experts model with ~3B active parameters). The Hugging Face model card confirms it: these aren’t Transformers or MLX checkpoints. They’re fixed-layout binaries that only load in Splash.
The engine’s kernels, draft model, and memory plan are all specialized for the exact shapes and architecture of each model. Fused shape-specific kernels, a trained DFlash 2 draft model, an optimized memory plan with pinned performance baselines. Inco even uses in-house kernel agents to write and tune Metal kernels for each model’s precise dimensions, things like decode kernels that read the 8-bit KV cache directly and reuse it across query heads and verification tokens, and GDN prefill kernels that keep recurrent state on-chip.
This is the same technology that powers their data-center inference platform, which claims to be the fastest provider on Artificial Analysis for five frontier open models. Now it’s coming to a laptop near you, under Apache-2.0.
The Numbers That Matter
The most compelling comparison isn’t just about raw decode speed, it’s about what happens when an agent actually works. Inco measured four parts of the agentic loop:
Decode (how fast tokens stream back as context grows):
| Model | Splash (short) | Splash (32K) | Next fastest | Ollama (32K) |
|---|---|---|---|---|
| Qwen3.8-27B | 74 tok/s | 54 tok/s | oMLX: 28 | 19 |
| Qwen3.6-35B-A3B | 210 tok/s | 143 tok/s | oMLX: 83 | 57 |
Cache reuse (the common case for agents, same context plus a few new tokens):
- Splash returns the first token in 282 ms on a cached 32K prompt replay for the 27B
- oMLX takes 2,049 ms, that’s 7.3× slower
- Ollama’s cache didn’t even hit on long prompts
Concurrency (when a fan-out agent spawns 4 parallel requests):
- Qwen3.8-27B: Splash hits 170 tok/s combined vs oMLX’s 43 (3.9× advantage)
- Qwen3.6-35B-A3B: 357 tok/s vs oMLX’s 177
The prefill numbers are also strong, about 2,000 input tokens/s on the 35B at 32K prompts, cutting the wait for first token from 29 to 17 seconds compared with oMLX. But here’s the honest caveat: reading a large repository cold is still expensive for every engine tested. Splash just makes it less painful.
The Elephant in the Room: Quantization
Now for the part that generated real debate on Reddit when the launch hit: Splash uses flat 4-bit quantization for all weights with an INT8 KV cache. No dynamic quantization, no hybrid precision, no per-layer tuning.
The criticism comes from a legitimate place. Qwen3.8-27B’s GDN layers are very sensitive to quantization, a point made by the developer of MTPLX, an alternative Apple silicon engine, who notes that his own implementation tunes certain layers to 8 or even 16 bits. Flat 4-bit across the board is a bold choice, and one that the Reddit crowd was quick to call out.
Inco’s counterargument: the draft model handles the speculation, the target verifies every drafted token, and speculation “changes speed and not the output distribution.” That’s technically true, the verified outputs match what the 4-bit model would produce. But it sidesteps the deeper question of whether a flat 4-bit Qwen3.8-27B is actually good at what you need it to do, especially for coding where precision matters.
Independent reviewers have been more measured. The MTPLX developer called Inco’s benchmarks “transparent” and noted the engine doesn’t take obvious shortcuts, while also flagging that the published tests use coding prompts where a DFlash draft is accepted most often. Speed on other kinds of text is the number to watch.
The Memory Math: Why 36 GB Isn’t Really Enough
Here’s where things get practical. Splash requires M3 or newer, macOS 26.4+, and 36 GB of unified memory minimum. Inco recommends 48 GB. That’s not marketing fluff, it’s arithmetic.
The Qwen3.8-27B package breaks down as:
– 14.1 GiB of 4-bit target weights
– 1.2 GiB DFlash 2 draft model
– 0.9 GiB vision encoder (yes, it handles images and inline PDFs)
That’s ~16.2 GiB before any KV cache. On a 36 GB Mac, macOS typically gives the GPU access to roughly 65, 75% of unified memory, so about 23, 27 GB. Subtract the model and draft, and you’re left with maybe 6, 10 GB for cache. At 32K context that’s workable but tight. On a 48 GB machine, you get roughly 14, 19 GB of cache headroom, which is why Inco recommends it: “it leaves room for an editor and a browser while a task runs.”
The memory plan is genuinely clever though. Because the model’s sizes are fixed and known in advance, Splash computes its budget at startup: Metal’s recommended working-set limit minus weights, draft, and per-request state. It prints the budget and stops if the model doesn’t fit. There’s no config file, no tuning, just --max-memory and --max-context ceilings for the edge cases.
This design pays off at the concurrency edge too: Inco sent 16 concurrent 32K requests to the 27B on a 48 GB machine. A general-purpose memory policy accepted nine on its first pass. Splash completed all 16, because it can budget nearly all available memory safely when it knows exactly what each request costs.
The Agentic Use Case: Where It Actually Wins
The most interesting part of Splash is how it’s designed for the way people actually use local models in 2026, as the backend for coding agents. The splash CLI has first-class integration with Claude Code, OpenCode, Codex, and Hermes built in. One command, no configuration:
brew install incoai/tap/splash
splash serve --model incoai/Qwen3.8-27B-Splash
Then from another terminal:
splash opencode # or: splash claude / splash codex / splash hermes
The API compatibility is impressive: OpenAI Chat Completions and Responses, plus Anthropic Messages, with streaming, tool calls, JSON Schema output, images, and inline PDFs. There’s even a thoughtful touch in the chat template: a system message after the first turn renders in place instead of being rejected, which coding agents that inject instructions mid-conversation depend on.
LM Studio picked it up on day zero too, the Bionic version (1.1.5+) has Splash as an experimental backend under Settings → Runtime. For non-terminal users, that’s a meaningful lowering of the barrier.
What This Means for the Broader Landscape
Splash is evidence of two converging trends. First, the efficiency gains from smaller, specialized AI models are real, DFlash 2 drafts have been downloaded over 6 million times and run in SGLang, vLLM, TensorRT-LLM, and llama.cpp. Second, the days of treating inference engines as generic model runners may be numbered. Just as GLM’s custom inference infrastructure is winning on data-center GPUs by building for specific workloads, Splash is proving the same principle applies on consumer hardware.
The pattern is consistent: specialization beats generalization when you control the full stack. The question is whether the ecosystem can sustain this level of per-model engineering. Splash requires custom kernels, trained drafts, and memory plans for every model it supports. That’s a model zoo in reverse, instead of one engine running many models, it’s many engines, each built for one model.
The Rust angle is relevant here too. NVIDIA’s Dynamo and HuggingFace’s Grout are both pushing toward safer, more specialized inference infrastructure. Splash’s automatic kernel generation via in-house agents is another data point in the same direction: handwritten kernels are too slow to produce and maintain at the pace the industry demands.
The Bottom Line
Splash is a genuine technical achievement. The decode numbers are real, the methodology is transparent, and the agentic use case is well-designed. On a 48 GB M5 Pro, an agent session deep into a 32K context runs faster on Splash than a fresh session does on any other engine tested.
But “built around the model” cuts both ways. You can’t load your own fine-tunes. You can’t switch to a different quantization. You can’t try a non-Qwen model. And the flat 4-bit weights are a legitimate concern for anyone using this for serious coding work where quantization error compounds over long sessions.
The 144 tok/s headline on the M5 Max is impressive, but the more meaningful number might be the 282 ms cached time-to-first-token, because that’s the difference between an agent that feels like it’s thinking and one that feels like it’s working.
Splash is not going to replace Ollama for the “I want to run whatever model I downloaded this week” crowd. But for the growing cohort of developers who spend their days with Claude Code or OpenCode pointed at a local endpoint, the 3.9× concurrency advantage when their agent fans out into sub-agents is the kind of difference that turns a frustrating session into a productive one.
It’s Apache-2.0, it’s on GitHub, and it’s free. The catch is real, but it’s also narrow: Splash is the fastest engine you can’t customize. For now, that’s a trade worth making, especially if you’re willing to wait for the 8-bit weights that the community is clearly asking for.
Images: Screenshot of the Splash inference engine interface showing performance metrics.