Commit graph

5 commits

Author SHA1 Message Date
ac70c57c15 chore(halo): preload both llama models and tune preset
Preload Qwen3.6-27B and Qwen3.6-35B-A3B at startup (load-on-startup)
so both are warm immediately under --models-max 2, set parallel = 1
as the [*] fallback for any other model, and adjust per-model context
size and draft depth.
2026-05-20 07:14:26 +02:00
31e491e314 Revert "fix(halo): 27 only"
This reverts commit 72e7bf613f.
2026-05-20 07:05:27 +02:00
72e7bf613f fix(halo): 27 only 2026-05-20 02:14:08 +02:00
807a3d0d8e fix(halo): context 2026-05-20 01:21:10 +02:00
0edf975c30 feat(halo): serve multiple llama models via models.ini preset
Replace the per-model llama-server units with a single service that
uses llama-server's --models-preset (models.ini) and --models-max 2,
so the 35B-A3B and 27B models are loaded on demand from one config.

Drop the now-redundant 27B / 27B-MTP / coder-next variant files and
the unused CacheDirectory + slot-save-path KV-slot handling.
2026-05-20 00:23:50 +02:00