Tagged with

4 articles found

GLM-4.7-Flash’s CUDA Fix: When Flash Attention Was the Problem, Not the Solution

A critical CUDA fix for GLM-4.7-Flash in llama.cpp reveals how a performance optimization was actively sabotaging local inference speeds, and why the community had to rebuild the wheel to make it work.

#cuda#Flash Attention#GLM-4.7...

amd

Cloud AI’s Worst Nightmare: 8 ‘Obsolete’ AMD GPUs Just Delivered 26.8 tok/s for $880

A community-built 8x AMD MI50 setup achieves production-grade LLM inference at a price that makes cloud providers nervous. Here’s how they pulled it off, and why the ‘graveyard GPU’ narrative is officially dead.

#amd#cost-optimization#GLM-4.7...

agentic workflows

GLM-4.7-Flash: The Local LLM That Actually Does What It Promises (Mostly)

GLM-4.7-Flash is delivering reliable agentic performance on consumer hardware, but the path to getting it running reveals the messy reality of local AI deployment.

#agentic workflows#GLM-4.7#llama.cpp...

GLM-4.7

Unsloth’s 2-Bit Miracle: How GLM-4.7 Lost 266GB Without Losing Its Mind

Unsloth’s aggressive 2-bit quantization slashes GLM-4.7 from 400GB to 134GB, forcing a reckoning with what ‘good enough’ means for frontier models

#GLM-4.7#local AI#model compression...