Gan, Jimmy 3994e29cb0 Initial: llama.cpp router mode optimization guide for Apple Silicon ...

Full documentation of optimizing two LLMs on a single M5 Max GPU:
- KV cache quantization (Q4_0)
- Flash attention and batch tuning
- Router mode with --models-max 1
- Per-model thread optimization via INI presets
- Before/after benchmarks (12→48 t/s on 27B, 23→132 t/s on 35B)

2026-07-10 02:27:36 +08:00

..

results.md

Initial: llama.cpp router mode optimization guide for Apple Silicon

2026-07-10 02:27:36 +08:00