# The Problem: Two Models, One GPU

## Why Two Models?

Different model architectures excel at different tasks:

- **MoE (Mixture of Experts):** Fast generation (~3B active params), good for chat. Example: Qwen3.6-35B-A3B
- **Dense:** Better reasoning on coding tasks, all parameters active. Example: Qwen3.6-27B

Running both gives you the best of both worlds — but on a single GPU, they compete.

## The Naive Approach

The obvious solution: run two `llama-server` processes on different ports.

```bash
# Server 1: 35B chat model on :8085
llama-server -m qwen35b.gguf --port 8085 -ngl 99

# Server 2: 27B coding model on :8080
llama-server -m qwen27b.gguf --port 8080 -ngl 99
```

**Result:** Both models stay permanently loaded in GPU memory, permanently competing for memory bandwidth.

## Why It's Slow

On Apple Silicon's Unified Memory architecture, GPU and CPU share the same memory pool. When two processes both use Metal GPU acceleration:

1. **Memory bandwidth is shared** — Both models' weights (75 GB combined) compete for the ~614 GB/s memory bus
2. **GPU scheduler splits time** — macOS Metal driver context-switches between processes
3. **KV cache doubles the tax** — Both models maintain large KV caches for their context windows

The result: each model gets roughly **half** the GPU bandwidth it could achieve alone.

## Measured Impact

| Model | Solo Speed | With Other Loaded | Penalty |
|-------|-----------|-------------------|---------|
| 35B MoE | 94 t/s | 23 t/s | **-75%** |
| 27B dense | 12 t/s | 12 t/s | **~0%** (already bandwidth-saturated) |

The 27B barely notices because it's already reading 27 GB of weights per token — its 12 t/s saturates the available bandwidth regardless. The 35B MoE (3B active params) has huge headroom but loses most of it to contention.

## The Goal

Find a way to give each model **full GPU bandwidth** without having to manually kill and restart servers.