There’s a particular kind of magic when someone posts a GIF of an AI model running on a phone and the comments section starts doing mental math about what it means for the cloud AI industry. That’s exactly what happened when a developer showed LFM2.5-2.6B, a 2.69B parameter agentic model, running at 17 tokens per second on a OnePlus 13 using nothing but the CPU.
No GPU. No NPU. No cloud round-trip. Just a 450KB inference engine built from scratch, a Q4_K_M GGUF quantization, and enough stubbornness to push the boundaries of what “edge AI” actually means.
The 17 tok/s That Got Everyone Talking
Let’s put that number in perspective. The developer, running their own inference engine through ADB, hit 17 tok/s on a phone that costs less than a mid-range GPU. That’s not benchmark-padding territory, that’s a real device, real CPU inference, and a model that’s been purpose-built for multi-step agent workflows with a 128K context window.
The engine itself is worth a closer look. The whole thing weighs in at 450KB and doesn’t just support LFM2.5-2.6B, it handles Qwen, Gemma, Bonsai, and other model architectures. The developer is already targeting ~30 tok/s, which would match what Liquid AI claims the model can sustain on phone-class hardware.
Seven teen tokens per second might not sound like much when you’re used to watching frontier models generate at 100+ tok/s on cloud GPUs. But here’s the thing: those tokens are free. They’re private. They work on a plane, in a basement, or in a country with restrictive data laws. And when the per-token cost hits zero, the economics of agentic AI change fundamentally.
Why This Model Is Different From the SLM Pack
Small language models have been the “next big thing” for years now. The problem? Most of them feel like a compromise, you get speed and privacy, but you lose the ability to actually do anything useful. The model can chat with you, sure, but ask it to plan a multi-step task or call a tool and it starts hallucinating faster than you can say “temperature 0.7.”
LFM2.5-2.6B takes a different approach. It’s not trying to be a smaller GPT-4. It’s built specifically for agentic workloads, and LFM2.5 as a lightweight, multimodal on-device foundation model sets the stage for what that means in practice.
The training pipeline tells the story. Four stages: supervised fine-tuning, teacher specialization, multi-domain on-policy distillation, and finally, agentic reinforcement learning. That last stage is where things get interesting. The model was trained inside real agent harnesses, Hermes Agent, OpenClaw, and Pi, learning to work with their tools, system prompts, and interaction patterns. This isn’t a model that was told how to call tools, it’s a model that practiced calling tools in production-like environments.
The result is a model that punches well above its weight class. Look at the numbers from Liquid AI’s release post:
| Benchmark | LFM2.5-2.6B (2.6B) | gemma-4-E4B-it (8B) | Qwen3.5-9B (9.7B) |
|---|---|---|---|
| Multi-IF | 80.07 | 77.35 | 62.55 |
| IFStruct | 85.49 | 76.65 | 78.50 |
| ToolSandbox | 77.83 | 65.00 | 76.44 |
| Claw-Eval avg (EN) | 62.85 | 58.02 | 66.53 |
A 2.6B model outperforming models three to four times its size on instruction following and tool use isn’t just impressive, it’s the kind of result that makes you question the “bigger is always better” orthodoxy. The liquid AI’s efficient MoE architecture enabling high-performance on-device inference from their larger models clearly carried over into this design.
The Architecture: Where the Efficiency Actually Comes From
Parameter count alone doesn’t explain this. The architecture does. LFM2.5-2.6B uses 30 layers split into 22 double-gated short convolution blocks and 8 grouped-query attention layers. It’s the LFM2 hybrid design, the same one that powers their larger models, applied at a scale that fits in your pocket.
The model was pre-trained on ~34T tokens, which is an absurd amount of compute for a model this size. More importantly, the tokenizer was extended in place to handle a 128K vocabulary, and mid-training included a dedicated context-extension phase pushing the context window to 128K.
The Phone Question: What 17 tok/s Actually Buys You
Here’s where we get to the real debate. The Reddit thread had a skeptic asking the obvious question: “In Artificial Analysis, it scores 3 intelligence points, the same as Qwen3.5 0.8b. So what’s the point if it performs like a model three times smaller?”
It’s a fair question, and the developer’s response cuts to the heart of it: “It’s focused on multi-step workflows meaning it’s just a very efficient part of an agentic loop.”
That’s not a dodge, it’s the entire thesis. A 17 tok/s model running locally isn’t meant to be your only interaction point with AI. It’s meant to be the worker in a loop that plans, calls tools, checks results, and iterates. At that speed, a complex task that requires 2,000 tokens of reasoning takes about two minutes. Clustered across a swarm of agents, each working on their own piece, you can parallelize in ways that cloud APIs structurally prevent.
Translation: this model can handle the long inputs, tool definitions, and multi-turn conversations that agentic workloads require. The context window isn’t a marketing bullet point, it’s the difference between a model that can follow a complex instruction and one that forgets what it’s doing halfway through.
The efficiency gains show up in the inference benchmarks. On an Apple M5 Max, the model decodes at 220 tok/s. On a Ryzen AI Max+ 395, it hits 113 tok/s. Apple’s memory bandwidth advantage for running large language models on consumer hardware like Apple Silicon is well documented, but seeing those numbers with a model that fits under 2.5GB is still remarkable.
The AI Weekly analysis nails the economic angle: “Removing the per-token cost changes how developers build: agents can now be massively parallelized on local hardware, running background tasks at no marginal cost.”
When token spend is no longer a constraint, you don’t have to think about whether an agent can afford to make 50 tool calls to solve a problem. You just let it work. The distilled small models outperforming larger frontier models in efficiency trend shows this isn’t an isolated case, either.
Not Everything Is Sunshine and On-Device Agents
Before we crown LFM2.5-2.6B the king of edge AI, let’s acknowledge the caveats.
The benchmarks are Liquid AI’s own selection against their choice of comparators. The model card itself notes it’s not recommended for coding-heavy or complex agentic tasks. And there’s already community feedback about hallucination issues when tools return malformed data, the model confidently presented incorrect information pulled through an MCP tool as fact, which is the classic small-model failure mode.
One HF community member raised the question directly: “When an MCP tool returns bad or wrongly formatted data, whose job is it to catch it, the model, the prompt, or the tool layer?” It’s a genuinely thorny problem. In a local agent, there’s no cloud API to blame or fallback to. The model and the harness have to handle it.
The licensing question also deserves attention. LFM2.5-2.6B ships under Liquid’s lfm1.0 license rather than a fully permissive open license. Developers with commercial plans should read the terms carefully before building on it.
And about the performance claims, the phone numbers in Liquid’s materials say “~30 tok/s” without naming a specific handset. The community’s 17 tok/s on a OnePlus 13 is a real datapoint, but it’s also a flagship phone with high-end silicon. Budget devices will be slower.
Setting Up Your Own Local Agent: It Takes Two Steps
For all the analysis, the practical part is refreshingly simple. Liquid’s guidance for running a local agent with LFM2.5-2.6B:
- Serve the model behind an OpenAI-compatible endpoint
- Point your agent harness (Hermes Agent, OpenClaw, or Pi) at it
Here’s what that looks like with the Hugging Face transformers integration:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "LiquidAI/LFM2.5-2.6B"
model = AutoModelForCausalLM.from_pretrained(
model_id,
device_map="auto",
dtype="bfloat16",
# attn_implementation="flash_attention_2" # uncomment on a compatible GPU
)
tokenizer = AutoTokenizer.from_pretrained(model_id)
prompt = "Plan and execute a multi-step task to organize my research notes"
input_ids = tokenizer.apply_chat_template(
[{"role": "user", "content": prompt}],
add_generation_prompt=True,
return_tensors="pt",
tokenize=True,
).to(model.device)
output = model.generate(
input_ids,
do_sample=True,
temperature=0.2,
top_k=80,
repetition_penalty=1.05,
max_new_tokens=512,
)
print(tokenizer.decode(output[0], skip_special_tokens=False))
Day-one support spans llama.cpp for GGUF edge inference, MLX for Apple Silicon, vLLM and SGLang for production serving, and ONNX for cross-platform deployment. Hardware support covers AMD, Qualcomm, Apple, NVIDIA, and Intel.
The Real Disruption: Zero Marginal Cost Changes Everything
The most interesting takeaway from this release isn’t the 17 tok/s on a phone. It’s what the Mamba 3’s inference-first design for efficient AI on local devices and Liquid’s approach have in common: the recognition that inference efficiency is the new battleground.
Cloud AI companies charge per token because inference costs real money at scale. LFM2.5-2.6B doesn’t eliminate that cost, it moves it. When inference happens on your phone or laptop, the marginal cost approaches zero. The electricity to run a model for an hour costs cents, not dollars.
The comment in that X thread put it bluntly: “A lot of companies have no idea that on-the-phone agents will be their nemesis soon.”
That’s not hyperbole. Enterprises running AI-powered workflows on private data have two options today: send everything to a cloud API and hope the vendor’s privacy policy is solid, or run models locally and actually control the data. For years, option two meant sacrificing capability. We’re at the point where that’s no longer strictly true.
LFM2.5-2.6B is doing something important, it’s not designed to replace cloud AI entirely, but to handle the high-volume, privacy-sensitive, low-latency tasks that make sense on the edge, while Apple M5 Max advancements in memory bandwidth for local LLM inference push the boundaries of what consumer hardware can handle. The cloud becomes the specialist, not the default. The phone becomes the workhorse.
What Actually Matters Here
The 17 tok/s number will be beaten. Someone will hit 30, then 40, then whatever the next flagship phone can manage. But the pattern is what matters. A sub-3B model that beats 9B models on tool use, runs on a phone, and costs nothing at inference time changes the calculus for every developer building agentic systems.
Skeptics will point to the benchmark selection, the license, the hallucination issues. They’re not wrong. But they’re examining the leaves while the forest burns.
The forest is this: edge AI just crossed a threshold. The “it’s only good for demos” era of on-device agents is over. Now it’s a production story, and the infrastructure is all there. The implications for hosted API providers, for data privacy, and for the entire economics of AI deployment are only starting to be recognized.
One thing is clear though. The next time someone posts a GIF of “quantization fidelity and benchmarking for selecting optimal GGUF models” running some tiny model on their phone, it won’t be a neat trick anymore. It’ll be the standard.
That’s what 17 tok/s on a phone really means: The AI you thought you needed the cloud for now runs in your pocket. And the agentic workflows running on it are about to change the way you build.
