Everything on llama.cpp
2 mentions, newest first.
tandem.cpp update adds AMD support and speeds up big MoE models on small GPUs
A Reddit user updated tandem.cpp, a patch set for llama.cpp that keeps hot experts on the GPU and lets the CPU help with the rest. The update adds AMD (ROCm) support and runs DeepSeek-V4.1 on a 64 GB PC with a 16 GB card. On DeepSeek-V4.1-Flash IQ2_XS with an RTX 5070 Ti, decode was 7.4 tok/s on llama.cpp and 14.2 tok/s on tandem.cpp.
Usable todayllama.cpp and ggml reach v1.0.0 with semantic versioning
The llama.cpp and ggml libraries (libllama and libggml) reach v1.0.0. API and ABI incompatibilities will now be signalled with major-version bumps. The release supports more than 150 architecture variants and all mainstream speculative decoding methods.
Usable today