Tagged with

3 articles found

Tencent Shrinks a 1.5TB Model to 200GB Without Breaking It, And the AI World Is Freaking Out

Tencent compressed Hy4-preview from 1.5TB to just 200GB in GGUF format while keeping 98% performance. Here’s what that means for local AI, inference costs, and the open-weight race.

#gguf#Hy4-preview#model compression...

AI Inference

The 3-Bit Gauntlet: How Extreme Quantization Is Reshaping AI Economics

Analysis of TurboQuant’s 6x compression breakthrough and Flash-Moe’s 397B parameter feat, exploring what extreme quantization means for distributed inference and edge deployment.

#AI Inference#Edge AI#model compression...

GLM-4.7

Unsloth’s 2-Bit Miracle: How GLM-4.7 Lost 266GB Without Losing Its Mind

Unsloth’s aggressive 2-bit quantization slashes GLM-4.7 from 400GB to 134GB, forcing a reckoning with what ‘good enough’ means for frontier models

#GLM-4.7#local AI#model compression...