feat(halo): add llama-server systemd unit for Qwen3.6-35B-A3B
Runs llama.cpp's ROCm build under DynamicUser, with the HF model cache in StateDirectory (survives systemctl clean) and KV slot saves in CacheDirectory. Listens on :8000. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
603e435db8
commit
b11e5c8356
2 changed files with 61 additions and 0 deletions
|
|
@ -10,6 +10,7 @@ with lib.metacfg;
|
|||
./hardware-configuration.nix
|
||||
#./xremap.nix
|
||||
./wyoming.nix
|
||||
./llama-server.nix
|
||||
];
|
||||
|
||||
boot.lanzaboote.pkiBundle = "/var/lib/sbctl";
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue