2 Commits

Author	SHA1	Message	Date
Gan, Jimmy	cc10540e9e	docs: add lessons-learned with all optimization insights ... Covers: launchd management, Dell hub power, batch tuning, thread count per arch, KV cache quantization, MTP speculation, flash attention, router mode vs two-servers, context sizing, swap monitoring.	2026-07-10 02:33:30 +08:00
Gan, Jimmy	3994e29cb0	Initial: llama.cpp router mode optimization guide for Apple Silicon ... Full documentation of optimizing two LLMs on a single M5 Max GPU: - KV cache quantization (Q4_0) - Flash attention and batch tuning - Router mode with --models-max 1 - Per-model thread optimization via INI presets - Before/after benchmarks (12→48 t/s on 27B, 23→132 t/s on 35B)	2026-07-10 02:27:36 +08:00