Tagged with

2 articles found

Who Needs a GPU Cluster? The Bare-Knuckle Reality of Training LLMs on a Single Card

Forget the H100s. We’re building capable transformers on a 5080 at home, diving into the trenches of data pipelines, gradient checkpointing, and the democratization of AI.

#machine learning#pytorch

avx2

20x Faster Top-K Sampling Without a GPU: The AVX2 Optimization Rewriting LLM Inference Rules

A new open-source AVX2-optimized Top-K implementation achieves 20x speedup over PyTorch CPU, delivering 63% faster prompt processing in llama.cpp for large MoE models, sometimes matching CUDA performance without the GPU overhead.

#avx2#cpu-optimization#llama-cpp...