Initial: llama.cpp router mode optimization guide for Apple Silicon

Full documentation of optimizing two LLMs on a single M5 Max GPU:
- KV cache quantization (Q4_0)
- Flash attention and batch tuning
- Router mode with --models-max 1
- Per-model thread optimization via INI presets
- Before/after benchmarks (12→48 t/s on 27B, 23→132 t/s on 35B)

docs/01-problem.md

@@ -0,0 +1,47 @@
+# The Problem: Two Models, One GPU
+
+## Why Two Models?
+
+Different model architectures excel at different tasks:
+
+- **MoE (Mixture of Experts):** Fast generation (~3B active params), good for chat. Example: Qwen3.6-35B-A3B
+- **Dense:** Better reasoning on coding tasks, all parameters active. Example: Qwen3.6-27B
+
+Running both gives you the best of both worlds — but on a single GPU, they compete.
+
+## The Naive Approach
+
+The obvious solution: run two `llama-server` processes on different ports.
+
+```bash
+# Server 1: 35B chat model on :8085
+llama-server -m qwen35b.gguf --port 8085 -ngl 99
+
+# Server 2: 27B coding model on :8080
+llama-server -m qwen27b.gguf --port 8080 -ngl 99
+```
+
+**Result:** Both models stay permanently loaded in GPU memory, permanently competing for memory bandwidth.
+
+## Why It's Slow
+
+On Apple Silicon's Unified Memory architecture, GPU and CPU share the same memory pool. When two processes both use Metal GPU acceleration:
+
+1. **Memory bandwidth is shared** — Both models' weights (75 GB combined) compete for the ~614 GB/s memory bus
+2. **GPU scheduler splits time** — macOS Metal driver context-switches between processes
+3. **KV cache doubles the tax** — Both models maintain large KV caches for their context windows
+
+The result: each model gets roughly **half** the GPU bandwidth it could achieve alone.
+
+## Measured Impact
+
+| Model | Solo Speed | With Other Loaded | Penalty |
+|-------|-----------|-------------------|---------|
+| 35B MoE | 94 t/s | 23 t/s | **-75%** |
+| 27B dense | 12 t/s | 12 t/s | **~0%** (already bandwidth-saturated) |
+
+The 27B barely notices because it's already reading 27 GB of weights per token — its 12 t/s saturates the available bandwidth regardless. The 35B MoE (3B active params) has huge headroom but loses most of it to contention.
+
+## The Goal
+
+Find a way to give each model **full GPU bandwidth** without having to manually kill and restart servers.