perf(llama-cpp-rocm): tune for Strix Halo (gfx1151)

- Restrict rocmGpuTargets to gfx1151 (Radeon 8060S, RDNA 3.5) — smaller
  closure, faster compile, no wasted device kernels.
- Enable GGML_HIP_ROCWMMA_FATTN: rocWMMA-backed flash attention is a
  major win on RDNA3+ for the GPU-offloaded attention path.
- Enable GGML_HIP_GRAPHS to lower per-token launch overhead.
- Add rocwmma to buildInputs to satisfy the WMMA path.

llama-server on halo runs with -ngl 99 --flash-attn on, so these flags
target the hot path. CPU-side AVX-512 was skipped intentionally — Zen 5
has it, but with full GPU offload the CPU paths barely run.
This commit is contained in:
Harald Hoyer 2026-05-06 09:13:54 +02:00
parent 623a71f95f
commit f62e8ac470

View file

@ -12,18 +12,33 @@ final: prev: {
# nodejs_20 # nodejs_20
; ;
llama-cpp-rocm = channels.unstable.llama-cpp-rocm.overrideAttrs (_: { # Tuned for Strix Halo (Ryzen AI Max+ 395 / Radeon 8060S, gfx1151).
src = prev.fetchFromGitHub { llama-cpp-rocm =
owner = "am17an"; (channels.unstable.llama-cpp.override {
repo = "llama.cpp"; rocmSupport = true;
rev = "267f8afe857b7bd1a49e4fde9138ab0f7be36625"; rocmGpuTargets = [ "gfx1151" ];
hash = "sha256-VYvRjnNZpPE60wjpBVO1FbURMRRMg71sM5kBxiEkElk="; }).overrideAttrs
postFetch = '' (prevAttrs: {
echo -n "267f8af" > $out/COMMIT src = prev.fetchFromGitHub {
''; owner = "am17an";
}; repo = "llama.cpp";
npmDepsHash = "sha256-k62LIbyY2DXvs7XXbX0lNPiYxuYzeJUyQtS4eA+68f8="; rev = "267f8afe857b7bd1a49e4fde9138ab0f7be36625";
}); hash = "sha256-VYvRjnNZpPE60wjpBVO1FbURMRRMg71sM5kBxiEkElk=";
postFetch = ''
echo -n "267f8af" > $out/COMMIT
'';
};
npmDepsHash = "sha256-k62LIbyY2DXvs7XXbX0lNPiYxuYzeJUyQtS4eA+68f8=";
buildInputs = (prevAttrs.buildInputs or [ ]) ++ [
channels.unstable.rocmPackages.rocwmma
];
cmakeFlags = (prevAttrs.cmakeFlags or [ ]) ++ [
"-DGGML_HIP_ROCWMMA_FATTN=ON"
"-DGGML_HIP_GRAPHS=ON"
];
});
/* /*
gnome-remote-desktop = channels.unstable.gnome-remote-desktop.overrideAttrs (prevAttrs: { gnome-remote-desktop = channels.unstable.gnome-remote-desktop.overrideAttrs (prevAttrs: {