If you dropped $7,000 on a maxed-out Mac Studio or snagged a DGX Spark thinking you’d be running the latest frontier models, I have bad news. The open model ecosystem has left you in a no-man’s-land.
Here’s the landscape: the last few months have delivered spectacular small models like Qwen 3.6’s 27B dense and Gemma 4’s 31B, alongside absolute behemoths like GLM 5.2’s 744B MoE and Kimi 2.7’s trillion parameters. Notice something missing? The entire middle ground. There’s nothing fresh in the 80-160B range that actually takes advantage of the 80-128GB unified memory systems that thousands of enthusiasts and professionals bought specifically for local AI.
This isn’t a whine post. It’s a technical diagnosis of why this gap exists, why it shouldn’t, and a call for model makers to cook something for the people who paid for the hardware.
The Sweet Spot That Got Skipped
The hardware targets for current model releases are brutally simple. As one sharp commenter on a recent discussion put it, model sizes are dictated by enterprise hardware constraints, not consumer ones. A 27B Qwen fits neatly on an H100’s 80GB VRAM at Q4_K_M. A 31B Gemma? Same deal. The 600B+ monsters are for people with GPU clusters who don’t care about fitting on a single card.
Meanwhile, the unified memory crowd, Apple Silicon users with 96GB+, AMD Ryzen AI 395 boxes, DGX Spark owners, and even folks with RTX 6000 Pros or four 3090s, are stuck with a Hobson’s choice. You either run older models like GLM 4.5 Air, GPT OSS 120B, or Qwen 3.5 122B that are now several generations behind the frontier, or you step down to the small 27-35B class models that don’t leverage your capacity.
This isn’t just about feeling left out. There’s a real performance ceiling here. The Qwen 3.5 122B that many DGX Spark users are running today? It got outclassed by its own smaller sibling, the Qwen 3.6 27B, in terms of quality per parameter. But running the 27B on a 128GB machine feels like buying a Ferrari and only driving it to the grocery store.
What a Proper 100B Model Would Look Like
The technical requirements for a killer mid-range model are well understood. We don’t need a dense 100B monster. What the community is screaming for is a sparse MoE in the 100B-120B total parameter range with an active parameter count around 10B.
Here’s why that specific recipe works:
- Total capacity (100-120B): Fits comfortably in 80-128GB of unified memory at FP4 or INT4 quantization. The weights alone would land around 60-70GB, leaving plenty of room for a solid KV cache and working context.
- Active parameters (~10B): This is the killer feature. With only ~10B active per token, the memory bandwidth bottleneck of unified memory systems becomes manageable. A modern M4 Max Mac Studio or a DGX Spark would see decode speeds in the 15-30 tokens-per-second range, which is more than usable for serious work.
- Latent KV cache: Using techniques like DeepSeek’s Multi-head Latent Attention, the KV cache footprint for long contexts collapses dramatically. A 128K context that would normally eat 40GB of VRAM compresses to under 10GB.
- Multi-token prediction: Lossless speculative decoding, already shipping in Gemma 4 and Qwen 3.6, would push effective throughput even higher with no quality hit.
A model like this wouldn’t just be acceptable, it would be transformative. It would slot directly into the hardware that thousands of people already own, giving them something that genuinely rivals closed-source APIs for most day-to-day tasks.
The Models We Have, and Why They’re Not Enough
Let’s be honest about the existing options. The commenters on the original thread had a mixed view of what’s currently available in the 100-120B range:
GLM 4.5 Air (106B-A12B): It’s technically the closest thing to what people want. But it’s an older generation architecture that struggles to match the intelligence density of Qwen 3.6’s smaller models. As one user put it, “they don’t want an older and larger model that does worse than Qwen 3.6 35B MoE.”
GPT OSS 120B: A community effort that’s been useful, but it hasn’t seen updates in months and the quality gap to newer models is widening.
Mistral’s recent 120B offerings: There was genuine excitement when Mistral dropped two new 120B-class models, one MoE and one dense. But early evaluations were brutal. One experienced evaluator wrote them off: “they don’t seem suitable for any purpose.”
Qwen 3.5 122B: This is the current workhorse for many Spark and Mac Studio users, but it arrived four months ago, and model years are like dog years. Its performance is now being comfortably matched or exceeded by models a quarter the size.
The pattern is clear. No one is currently training a mid-size MoE with modern training techniques, modern data mixes, and modern quantization awareness. The closest we’ve gotten is DeepSeek V4 Flash, but that requires dual GPUs and sits at a price point that defeats the purpose of unified memory.
Unified Memory Architecture: Accidental Perfect Fit
There’s an irony here that’s worth unpacking. The surge in MoE adoption wasn’t designed for unified memory systems. It was driven by datacenter economics: MoE models deliver more intelligence per FLOP, which is what matters when you’re renting compute by the hour.
But the engineering trade-offs of MoE happen to align almost perfectly with the constraints of Apple Silicon and similar unified memory architectures. As one analysis of local models in mid-2026 noted, “unified-memory machines turned out to suit these models almost by accident.”
Here’s the technical reason: MoE models have massive weight storage requirements but modest bandwidth needs. A 120B model with 10B active needs you to hold 120B parameters in memory, but you only read about 10B per token. Unified memory systems excel at this because they trade raw bandwidth for large capacity. An M4 Max’s memory bandwidth (~500 GB/s) is far slower than a 5090’s (~1.8 TB/s), but it can hold 128GB of data without splitting across PCIe links or dealing with NVLink overhead.
| Architecture | 100B MoE Model | 27B Dense Model |
|---|---|---|
| Memory needed (weights at INT4) | ~60 GB | ~17 GB |
| Active per token | ~5-10 GB | ~17 GB |
| Bandwidth bottleneck | Low (small active load) | High (full parameter read) |
| Ideal for unified memory? | Yes | Overkill on capacity, fine on speed |
When you put it in a table like that, the case for a mid-range MoE on unified memory becomes unassailable.
Hardware Is Only Getting More Expensive
Memory manufacturers have reallocated wafer capacity toward HBM, which earns several times more per wafer than conventional DRAM. The result has been brutal: conventional DRAM contract prices surged 90-98% quarter-on-quarter in early 2026. PC DRAM passed 100% increases. NAND followed. A 1TB SSD nearly doubled in price.
SK Hynix told an earnings call it had already sold out next year’s HBM capacity. Any genuine relief for consumers wanting to buy GPUs, RAM, or SSDs isn’t expected before late 2027.
So the people who already have 80-128GB of unified memory, the Mac Studio buyers, the DGX Spark early adopters, the Strix Halo enthusiasts, are sitting on hardware that cost a fortune and is now more valuable than ever. And the model ecosystem is treating them like second-class citizens.
The RTX Spark hardware coming this fall (announced with Microsoft at the end of May) will add more unified memory devices to the mix, with the promise of running 120B-parameter models at million-token context locally. But as someone who’s been running a DGX Spark since launch, I can tell you that promise feels hollow without the actual models to run on it.
The Deep Seats on the Bench: Inference Optimization Tailwinds
Before we descend entirely into doom, there have been genuine engineering breakthroughs that make this request feasible, not fantasy.
The inference optimization breakthroughs for large models we’re seeing from various teams are staggering. Four-bit quantization went from research curiosity to default shipping format. NVIDIA’s Blackwell does FP4 in hardware. OpenAI released gpt-oss natively in MXFP4.
Sparse attention, pioneered at scale by DeepSeek, drops the complexity of long-context inference from quadratic to roughly linear in the selected set. DeepSeek’s lightning indexer runs on a separate CUDA stream, hiding its latency behind work that’s already happening.
Multi-token prediction, which was proven at scale by DeepSeek-V3 and is now shipping in Gemma 4 and Qwen 3.6, delivers roughly 1.8x throughput gain with no quality loss. The catch? The gains depend on workload predictability. But for most use cases, that’s a meaningful speedup.
Combine all of this: sparse MoE architecture, FP4 quantization, compressed latent KV cache, and multi-token prediction inference. A well-trained 100B-A10B model from a 2026-era training run would absolutely demolodge anything currently available in its size class, and it would run beautifully on the hardware that already exists.
What We’re Actually Asking For
The community sentiment, distilled from dozens of comments and analyses, isn’t for something exotic. The request is painfully simple:
- Total parameters: 100-120B
- Active parameters: 10-12B
- Architecture: Sparse MoE
- Training data: Current generation (not Qwen 3.5 era)
- Quantization: Native FP4 support
- Context: Million-token latent KV
This isn’t asking for a miracle. This is asking for a model that aligns with the hardware that already exists and costs more than ever to acquire.
Some commenters pointed out that we’ve already seen Qwen3-Coder-Next, an 80B model with only 3B active parameters designed specifically for local development. That’s a step in the right direction, but it’s a specialized coder model, not a general-purpose intelligence upgrade.
The industry trend toward smaller, more accessible open models suggests someone should be paying attention to this gap. NVIDIA themselves owe it to DGX Spark customers, given the device’s rocky launch and continued support issues with NVFP4. They’ve delivered Nemotron 3 Super, but that’s also from an older generation and has been surpassed in quality by newer small models.
The Moonshot: Training a 100B MoE on Open Infrastructure
Here’s the part that gets interesting from a community perspective. Training a 100B-class MoE isn’t cheap, but it’s not insane either. The compute required is roughly 1/10th of what DeepSeek V3 cost to train. For comparison, that’s around 2-3 million GPU-hours on H100s.
At current spot pricing (~$3/GPU-hour), that’s $6-9 million. That’s not pocket change, but it’s in the range of what well-funded open-source projects and university collaborations could assemble. The real bottleneck isn’t the compute budget, it’s the organizational will. No major lab currently sees the unified memory segment as worth targeting, because the money is in cloud inference and enterprise deployments.
But the market of frustrated enthusiasts with expensive hardware is real and vocal. If someone does drop this model, they’ll capture a community that’s actively looking for reasons to stay local rather than retreating to API calls.
There’s also the possibility of using efficiently serving models of varying sizes from a unified checkpoint, which NVIDIA’s Star Elastic approach enables. Imagine a single training run that can produce 27B, 80B, and 120B variants from the same checkpoint. That would dramatically reduce the marginal cost of adding the mid-range option.
There are also reasons to run local AI models instead of relying on cloud services that go beyond convenience. The recent emergency export controls that forced Anthropic to withdraw Fable 5 and Mythos 5 overnight were a stark reminder that cloud API access can disappear at the stroke of a pen from a regulator. Local model availability isn’t just about speed, it’s about sovereignty.
The Way Forward
This isn’t going to fix itself. Model makers optimize for the largest market, and the largest market is still the datacenter. The unified memory segment, while enthusiastic and well-funded, doesn’t move the needle for companies spending tens of millions on training runs.
But the community has options. The efficient open-weight models that provide viable alternatives like MiniMax M3 show that it’s possible to compete without being the biggest. We need the same energy applied to the mid-range.
Individual actions that might move the needle:
- Signal demand. If you’re a Spark, Mac Studio, or Strix Halo owner, be vocal. Upvote discussions, write blog posts, make noise. Model makers pay attention to community heat.
- Support inference engine development. Ollama’s recent MLX engine preview for Apple Silicon is a good sign. The more efficient the runtime, the more feasible these models become.
- Collaborate on a community training effort. The open-source community has trained capable models before. The compute landscape is improving, and a targeted effort could close this gap.
- Push for adaptable architectures. If NVIDIA’s Star Elastic approach or similar techniques become standard, the marginal cost of serving multiple sizes drops to near zero. That’s the most realistic path to getting the mid-range option for free.
The good news? The engineering to make this work already exists. The bad news? Someone has to actually train the model.
Until then, we’re all owners of very expensive hardware running software that can’t fully use it. And that’s a waste this community shouldn’t accept.
