Tagged with

3 articles found

Apple’s Desperate AI Shortcut: Cramming a 27B Model Into Your iPhone

Apple is reportedly in talks with startup PrismML to compress a 27-billion-parameter AI model from 54GB to under 4GB, aiming to run GPT-class intelligence directly on iPhones. Here’s why skepticism is warranted.

#AI Compression#model quantization#On-Device AI...

distributed systems

1000 Tokens Per Second on a 1T Model? Xiaomi Just Broke Physics (or At Least the Latency Barrier)

Xiaomi’s MiMo v2.5 hits 1000 TPS on a trillion-parameter model using commodity GPUs. Here’s the deep dive on the FP4 quantization, DFlash speculative decoding, and TileRT systems alchemy that made it possible.

#distributed systems#Inference Optimization#Mixture of Experts...

1-bit models

Your Browser Just Became an Image Generation Engine: PrismML’s 3GB Model Changes Everything

PrismML’s Bonsai Image 4B compresses a 16GB model to 3GB using 1-bit quantization, running entirely in-browser on WebGPU. Here’s the technical breakdown and the controversy around attribution.

#1-bit models#image generation#local AI...