Cloud AI’s Worst Nightmare: 8 ‘Obsolete’ AMD GPUs Just Delivered 26.8 tok/s for $880

The cloud AI industrial complex has a dirty little secret: they’re terrified of what happens when developers realize they can replicate their services for the cost of a single month’s API bill. That terror just got a face, eight of them, actually. AMD MI50 GPUs, pulled from the digital scrap heap, are now pumping out 26.8 tokens per second on MiniMax-M2.1 and 15.6 tok/s on GLM-4.7, all while humming along in someone’s basement on a rig that cost less than a MacBook Pro.

The math is almost offensive. $880 for 256GB of VRAM. That’s $3.44 per gigabyte of GPU memory. Compare that to AWS’s p4d.24xlarge at $32.77 per hour for 320GB of A100 memory, and you’re looking at a break-even point measured in days, not years. This isn’t just a hobbyist flex, it’s a direct assault on the economics of cloud inference.

The $880 Beast: What You’re Actually Buying

• 8x AMD MI50 32GB (datacenter cards from 2018-2019, eBay’s finest)
• ASRock Rack ROMED8-2T motherboard with 7 PCIe 4.0 x16 slots
• AMD EPYC 7642 (48 cores, 2.3 GHz, 128 PCIe lanes)
• 64GB DDR4 3200 ECC RAM (yes, just 64GB system RAM for 256GB VRAM)
• Dual 1800W PSUs with an add2psu adapter
• 5x LINKUP Ultra PCIe 4.0 x16 risers and various SlimSAS adapters

The total GPU cost? $880 at early 2025 prices. The motherboard will run you another grand, which critics are quick to point out. But even at $1,880 total, you’re still in “aggressively affordable” territory for what amounts to a local AI supercomputer. The power draw is no joke, 280W idle, 1,200W under full inference load, but that’s the price of admission to the big leagues.

Performance Numbers That Break the Simulation

Here’s where things get spicy. The benchmarks from this setup don’t just compete, they’re actively embarrassing for cloud providers:

For context, that’s roughly half the speed of a single RTX 4090 on GLM-4.7, but with 8x the VRAM and at 1/10th the GPU cost. The author notes that parallel sequences for tools like Claude Code can nearly double the effective tok/s, pushing MiniMax-M2.1 into the 40-50 tok/s range for real-world agentic workflows.

The Software Stack: Duct Tape, ROCm, and Sheer Willpower

# Ubuntu 24.04 with ROCm 6.3.4
wget https://repo.radeon.com/amdgpu-install/6.3.4/ubuntu/noble/amdgpu-install_6.3.60304-1_all.deb
sudo apt install ./amdgpu-install_6.3.60304-1_all.deb
sudo amdgpu-install --usecase=rocm --rocmrelease=6.3.4

pyenv install 3.12.11
pyenv virtualenv 3.12.11 venv312
pyenv activate venv312

git clone --branch v3.5.0+gfx906 https://github.com/nlzy/triton-gfx906.git
cd triton-gfx906
pip install 'torch==2.9' --index-url https://download.pytorch.org/whl/rocm6.3
pip wheel --no-build-isolation -w dist .
pip install ./dist/triton-*.whl

git clone --branch v0.12.0+gfx906 https://github.com/nlzy/vllm-gfx906.git
cd vllm-gfx906
pip install -r requirements/rocm.txt
pip wheel --no-build-isolation -w dist .
pip install ./dist/vllm-*.whl

The critical detail? Patches are required for both GLM-4.7 and MiniMax-M2.1. The repository includes specific patch files that must be copied into your vLLM installation. This is where the community’s value becomes undeniable, without these patches, you’re running at a fraction of the performance or not at all.

The GLM-4.7 Saga: When a Single Line of Code Breaks Everything

The most revealing part of this story isn’t the hardware, it’s the software drama that unfolded in real-time. GLM-4.7-Flash shipped with a critical bug in its MoE gating function. The model was using softmax instead of sigmoid for expert routing, which is like putting diesel in a gasoline engine and wondering why it coughs.

The fix was a one-line change in convert_hf_to_gguf.py, but the impact was massive. Early adopters reported “garbage output” and “infinite loops” until the community identified the root cause. The llama.cpp PR #18980 that fixed this was merged on January 21, 2026, and suddenly GLM-4.7 went from “unusable” to “unbelievably strong for its size.”

Cost Analysis: The Cloud Rebellion’s Economic Foundation

That’s not a year. That’s not a quarter. That’s less than three weeks before your basement supercomputer pays for itself. And after that? Free inference, minus electricity.

The Parallel Processing Secret: KV Cache Magic

The author’s logs reveal the real performance multiplier: “GPU KV cache size: 409,280 tokens… Maximum concurrency for 196,608 tokens per request: 2.08x”

NCCL_P2P_DISABLE=1 VLLM_USE_TRITON_AWQ=1 OMP_NUM_THREADS=4 \
  vllm serve ~/llm/models/MiniMax-M2.1-AWQ \
    --tensor-parallel-size 8 \
    --max-model-len 196608 \
    --gpu-memory-utilization 0.95 \
    --enable-auto-tool-choice \
    --tool-call-parser minimax_m2

Graveyard GPUs vs. The Future: What This Actually Means

The MI50 is based on AMD’s Vega 20 architecture, released in 2018. These cards were designed for HPC workloads, not AI inference. They lack Tensor Cores, have “only” 7.4 TFLOPS of FP32 performance, and draw 300W each. By all conventional wisdom, they should be obsolete.

The Fine Print: What They Don’t Tell You

• Power and Cooling
  – 1,200W under load requires serious electrical infrastructure
  – Each MI50 needs dedicated cooling (the author uses 2x 50mm fans per GPU)
  – Your basement will sound like a data center
• Software Fragility
  – You’re running forks of forks. Updates break things.
  – The GLM-4.7 bug fix saga shows how quickly “working” becomes “broken”
  – No official support. Your best help is GitHub issues and Discord
• Motherboard Costs
  The ASRock Rack ROMED8-2T costs nearly as much as the GPUs. While consumer motherboards with PCIe splitters are possible, they’re experimental. This isn’t Lego, it’s electrical engineering.

Practical Implications: What You Can Actually Build

• Offline code agents that process entire codebases in context
• Document analysis pipelines that ingest 500-page PDFs
• Multi-agent workflows with parallel tool execution
• Private RAG systems that never leave your network

The Bottom Line: The Rebellion Is Here

This setup isn’t perfect. It’s loud, it’s power-hungry, and it requires constant maintenance. But it’s also a proof of concept that the economic foundation of cloud AI is crumbling.

Want to replicate this setup? The full configuration details are available in the GitHub repository. Fair warning: patience and a high tolerance for debugging required.