feat(halo): add llama-server systemd unit for Qwen3.6-35B-A3B

Runs llama.cpp's ROCm build under DynamicUser, with the HF model cache
in StateDirectory (survives systemctl clean) and KV slot saves in
CacheDirectory. Listens on :8000.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Harald Hoyer 2026-05-05 10:02:51 +02:00
parent 603e435db8
commit b11e5c8356
2 changed files with 61 additions and 0 deletions

View file

@ -10,6 +10,7 @@ with lib.metacfg;
./hardware-configuration.nix
#./xremap.nix
./wyoming.nix
./llama-server.nix
];
boot.lanzaboote.pkiBundle = "/var/lib/sbctl";