This isn’t just another big model. Inkling skips Rotary Position Embeddings (RoPE) entirely, sprinkles short convolution layers throughout the decoder, and adds an extra RMSNorm right after the embedding layer. Some of these decisions feel borderline heretical to the modern LLM orthodoxy. The conventional wisdom says you use RoPE, you avoid unnecessary normalization, and you keep your architecture clean. Inkling ignores all three.
The Architecture That Breaks the Mold
Let’s start with the bones. Inkling is a 66-layer decoder-only transformer with a sparse MoE backbone. Each MoE layer contains 256 routed experts plus 2 shared experts that fire on every token. Six routed experts activate per token, giving you 41B active parameters out of 975B total. That’s a 4.2% activation rate, less sparse than Kimi K2.5’s 3.2% (32B of 1T), but still in the ballpark.
But the real surprises are in the attention mechanism and the convolutional interjections.
No RoPE? No Problem?
The most eyebrow-raising decision is the use of a learned, input-dependent relative-position bias instead of RoPE. Every major open-weight model in the last two years has used RoPE, DeepSeek, Llama, Qwen, Mistral, all of them. It’s become the default positional encoding strategy because it extrapolates reasonably well to longer sequences and integrates cleanly with the attention mechanism.
Inkling throws that out. Instead, each attention layer learns a per-token, per-head relative feature via a fourth projection tensor beyond Q/K/V. This tensor is then modulated with distance information and propagated into the attention logits directly.
The lab claims this approach “performs better and extrapolates better to longer sequences” than RoPE. Sebastian Raschka noted in his analysis that the sliding-window-heavy architecture probably makes this work. Of the 66 decoder layers, 55 use local attention with a tiny 512-token window. Within those windows, a learned relative bias provides plenty of positional signal.
The 11 global layers apply the learned bias only over the preceding 1,024 tokens. Beyond that range, attention is effectively content-based with respect to position. This mirrors the intuition behind NoPE (No Positional Embeddings), which other architectures use selectively.
Is this actually better? The jury is out. The lab’s internal ablations suggest it works, but we need independent verification. What’s clear is that this is a deliberate, well-reasoned departure from the norm, not a mistake.
Short Convolutions: The Unexpected Guest
Here’s where things get really interesting. Every decoder layer applies short 1D convolutions with kernel size 4 after the key and value projections, and again on the attention and MLP branch outputs. These aren’t the state-space model convolutions you see in Mamba or Hyena, they’re tiny, cheap, and strategically placed.
The intuition is elegant: these SConv layers provide cheap local token mixing and an explicit short-range inductive bias. They free the attention and MoE modules from having to encode local patterns, letting them focus on longer-range dependencies and expert specialization. It’s a form of architectural load-balancing, give the cheap subnetwork the local work, let the expensive one handle the global stuff.
This is reminiscent of the hybrid Mamba-Transformer approach used by Nemotron 3 Ultra, but where Nemotron alternates between an RNN-like state-space layer and attention, Inkling inserts its convolutions as lightweight augmentations within the transformer itself. It’s a less radical departure but arguably more practical, you don’t need a completely new architecture class, just a few extra 1D conv kernels.
The Extra RMSNorm: Almost Redundant?
The config file reveals an additional RMSNorm directly after the token embedding layer, separate from the pre-attention RMSNorm inside every transformer block. At first glance, this looks like a mistake or a vestige of an experimental configuration that leaked into production.
But it’s explicitly enabled in the configuration and present in the Transformers implementation. Raschka speculated that it might be almost redundant, and he’s probably right, but that doesn’t mean it’s useless.
The embedding layer is where all modalities (text, image, audio) first converge into a shared hidden space. An extra normalization step here could help stabilize the combined signal before it enters the transformer stack. Given that Inkling uses encoder-free early fusion for multimodality, where images become 40×40 pixel patches through a four-layer hMLP and audio enters as discretized mel spectrograms, the extra normalization might be critical for handling the variance across modality projections.
Multimodal Without the Encoder
Speaking of multimodality, Inkling’s approach is refreshingly different from the bolt-on encoder pattern. Most multimodal models slap a separate vision encoder (like CLIP or SigLIP) onto a text model. Inkling doesn’t.
Instead, it uses lightweight embedding front ends for each modality. Images pass through a simple hierarchical MLP patchifier that progressively merges pixels until one embedding per patch remains. Audio is encoded as dMel spectrograms in 100 ms chunks. Both project into the same hidden space as text tokens, and the decoder processes them jointly.
This is encoder-free early fusion. There’s no separate vision tower, no dedicated audio encoder. The same 66-layer decoder learns to reason across all modalities from the ground up. The model was pretrained on 45 trillion tokens spanning text, images, audio, and video.
The trade-off is clear: you lose some modality-specific preprocessing and representation learning, but you gain a unified architecture that’s simpler to fine-tune and deploy. For domain adaptation, which is Thinking Machines’ entire thesis, this matters. A single model that you can fine-tune for any modality combination is more useful than a cascade of specialist models.
The Controllable Reasoning Dial
Inkling’s most immediately practical innovation is the controllable thinking effort. During reinforcement learning (which ran over 30 million rollouts), the team set effort by changing the system message and adjusting per-token cost. The model learned to spend different token budgets on different rollouts.
This is exposed in transformers (5.14.0+) as a reasoning_effort argument with named levels: "none", "minimal", "low", "medium", "high", "xhigh", and "max". The announcement sweeps effort from 0.2 to 0.99.
The efficiency data is compelling. Inkling uses one third as many tokens as Nemotron 3 Ultra to achieve equal Terminal-Bench 2.1 performance. Independent analysis from Artificial Analysis corroborates the pattern: Inkling averages 25K output tokens per Intelligence Index task, versus 37, 43K for GLM-5.2, Kimi K2.6, and DeepSeek V4 Pro.
For agentic workloads billed per token, this changes the cost calculus dramatically. A model that spends 25K tokens where rivals spend 37, 43K isn’t just efficient, it’s economically decisive. You’re buying outcomes per dollar, not points on a leaderboard.
Performance: Competitive, Not Dominant
Thinking Machines is refreshingly honest about where Inkling stands. Their launch post states explicitly: “Inkling is not the strongest overall model available today, open or closed.” This isn’t false modesty, the benchmarks back it up.
Against open-weight peers, Inkling leads on some metrics and trails on others:
| Benchmark | Inkling | GLM-5.2 | Nemotron 3 Ultra | Kimi K2.6 |
|---|---|---|---|---|
| HLE (text only) | 29.7% | 40.1% | 26.6% | 35.9% |
| AIME 2026 | 97.1% | 99.2% | 94.2% | 96.4% |
| GPQA Diamond | 87.2% | 89.5% | 86.7% | 91.1% |
| SWEBench Verified | 77.6% | 80.0% | 70.7% | 80.2% |
| Terminal-Bench 2.1 | 63.8% | 82.7% | 56.4% | 71.3% |
| SimpleQA Verified | 43.9% | 38.1% | 32.4% | 38.7% |
| IFBench | 79.8% | 73.3% | 81.4% | 76.0% |
| FORTRESS (Adversarial) | 78.0% | 71.3% | 77.6% | 65.6% |
Against closed frontier models like Claude Fable 5 and GPT-5.6 Sol, Inkling trails on most reasoning and coding benchmarks. But the gap isn’t embarrassing, it’s competitive enough to be worth customizing. The model’s FORTRESS Adversarial score (78.0%) leads the open-weight group, and its 77.1% on BrowseComp is respectable.
The real story might be the multimodal scores. Inkling posts 73.5% on MMMU Pro, 78.1% on CharXiv RQ (82.0% with Python), and 91.4% on VoiceBench. For a model that processes text, images, and audio natively without separate encoders, these numbers are impressive.
Beyond Raw Benchmarks: The Second-Order Effects
Inkling’s significance goes beyond its benchmark chart. It’s the first US-built open-weight model at this scale. Through the first half of 2026, “open weight” functionally meant “Chinese lab” for anyone running procurement, compliance, or export-control review. Multiple outlets describe Inkling as the largest American open-weight model released to date, surpassing Nvidia’s Nemotron 3 Ultra.
This matters for teams whose legal or security review blocks Chinese-origin weights. They now have a credible permissively-licensed alternative to evaluate. Even teams that never deploy Inkling benefit from the pricing and licensing discipline it introduces to vendor conversations.
The lab’s financial reasoning case study with Bridgewater Associates is a directional signal for the customize-don’t-rent thesis. Fine-tuning an open model on proprietary domain expertise yielded 84.7% on financial-reasoning tests at roughly one-fourteenth the cost of a top proprietary model. These are the lab’s own numbers, not independently verified, but the pattern is worth watching.
Running Inkling: Realistic Footprints
Let’s talk about actually running this thing. The BF16 checkpoint needs 2 TB of aggregated VRAM, eight NVIDIA B300 or sixteen H200 GPUs. The NVFP4 quantized checkpoint drops that to 600 GB, targeting four B300 or eight H200 systems.
For most organizations, the realistic on-ramps are hosted inference through TogetherAI, Fireworks, Modal, Databricks, or Baseten, or fine-tuning through Tinker. Day-zero support in SGLang, vLLM, llama.cpp, and Unsloth means you have options.
Unsloth’s Dynamic 1-bit GGUFs are particularly interesting: they’re 86% smaller than BF16 while retaining ~74.2% of top-1% accuracy, fitting into ~280 GB of combined RAM and VRAM. A Mac Studio Ultra or a carefully specced multi-GPU rig can host the smallest quant.
# Basic inference with Inkling using transformers
from transformers import AutoModelForMultimodalLM, AutoProcessor
model_id = "thinkingmachines/Inkling"
# model_id = "thinkingmachines/Inkling-NVFP4" # for Blackwell
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(
model_id, dtype="auto", device_map="auto",
)
messages = [
{"role": "user", "content": [
{"type": "image", "image": "chart.png"},
{"type": "text", "text": "Analyze this chart and explain the trend."},
]},
]
inputs = processor.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
reasoning_effort="medium",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=2000, use_mtp=True)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))
What Makes This Release Important
Inkling isn’t the strongest model on any single benchmark. It doesn’t claim to be. What it does offer is a credible, permissively-licensed, natively multimodal base model that organizations can inspect, modify, and fine-tune for their own workloads.
The architectural surprises, no RoPE, short convolutions, extra normalization, are the kind of heterodox choices that move the field forward. Even if they don’t all pan out, they demonstrate that the current architectural consensus isn’t the only viable path. The controllable reasoning dial and token efficiency are genuinely useful innovations.
For practitioners, the message is clear: ignore the vendor chart, run your own evals on your own tasks, and price self-hosting against metered APIs at your actual volume. Inkling’s independently measured token efficiency shifts the cost curve in its favor for agentic workloads. The niche for customizable open-weight models just got a lot more interesting.
