Young Tries AI

Everything on llama.cpp

2 mentions, newest first.

  1. tandem.cpp update adds AMD support and speeds up big MoE models on small GPUs

    A Reddit user updated tandem.cpp, a patch set for llama.cpp that keeps hot experts on the GPU and lets the CPU help with the rest. The update adds AMD (ROCm) support and runs DeepSeek-V4.1 on a 64 GB PC with a 16 GB card. On DeepSeek-V4.1-Flash IQ2_XS with an RTX 5070 Ti, decode was 7.4 tok/s on llama.cpp and 14.2 tok/s on tandem.cpp.

    Usable today
  2. llama.cpp and ggml reach v1.0.0 with semantic versioning

    The llama.cpp and ggml libraries (libllama and libggml) reach v1.0.0. API and ABI incompatibilities will now be signalled with major-version bumps. The release supports more than 150 architecture variants and all mainstream speculative decoding methods.

    Usable today