Tagged with

3 articles found

The 30B Raspberry Pi Breakthrough That Flips GPU Optimization on Its Head

Recent advances in quantization and kernel optimization are enabling 30B-parameter models to run on Raspberry Pi devices, but the real story is how they expose a fundamental flaw in our understanding of model compression: fewer bits doesn’t always mean faster inference.

#kernel-optimization#llama.cpp#quantization...

diffusion-models

Tencent’s WeDLM 8B: When Diffusion Models Beat Autoregressive LLMs at Their Own Game

Tencent’s diffusion-based language model achieves 3-6× faster inference than vLLM-optimized Qwen3-8B on math reasoning, challenging the token-by-token generation paradigm that has dominated LLMs since GPT-2.

#diffusion-models#llm-inference#math-reasoning...

cuda

llama.cpp’s Qwen3 Integration Pits Local AI Against the Cloud Giants

After months of development, Qwen3-Next is finally coming to llama.cpp with optimized CUDA operations, enabling fast local inference on consumer NVIDIA hardware.

#cuda#llamacpp#local-ai...