# Step 1: Two Separate Servers

The initial setup for running both model servers as persistent macOS services via `launchd`.

## Launchd Plist for 35B Chat Model

```xml
<!-- ~/Library/LaunchAgents/com.jimmyg.llama-server-qwen35b.plist -->
<plist>
<dict>
  <key>KeepAlive</key><true/>
  <key>ProgramArguments</key>
  <array>
    <string>/opt/homebrew/bin/llama-server</string>
    <string>-m</string>
    <string>/Users/jimmyg/models/Qwen3.6-35B-A3B-Q8_0.gguf</string>
    <string>--host</string><string>0.0.0.0</string>
    <string>--port</string><string>8085</string>
    <string>-ngl</string><string>99</string>
    <string>-c</string><string>262144</string>
    <string>--mlock</string>
  </array>
  <key>RunAtLoad</key><true/>
  <key>StandardOutPath</key>
  <string>/Users/jimmyg/.hermes/logs/llama-server-qwen35b.log</string>
  <key>StandardErrorPath</key>
  <string>/Users/jimmyg/.hermes/logs/llama-server-qwen35b.log</string>
</dict>
</plist>
```

## Launchd Plist for 27B Coding Model

```xml
<!-- ~/Library/LaunchAgents/com.jimmyg.llama-server.plist -->
<plist>
<dict>
  <key>KeepAlive</key><true/>
  <key>ProgramArguments</key>
  <array>
    <string>/opt/homebrew/bin/llama-server</string>
    <string>-m</string>
    <string>/Users/jimmyg/models/Qwen3.6-27B-Q8_0.gguf</string>
    <string>--host</string><string>0.0.0.0</string>
    <string>--port</string><string>8080</string>
    <string>-ngl</string><string>99</string>
    <string>-c</string><string>131072</string>
  </array>
  <key>RunAtLoad</key><true/>
</dict>
</plist>
```

## Initial Flags Explained

| Flag | Value | Purpose |
|------|-------|---------|
| `-ngl 99` | All layers on GPU | Full Metal acceleration |
| `-c` | 262K / 131K | Context window size |
| `--mlock` | - | Prevent model from being swapped |
| `--spec-type draft-mtp` | - | Multi-Token Prediction speculation |

## Loading the Services

```bash
launchctl load ~/Library/LaunchAgents/com.jimmyg.llama-server-qwen35b.plist
launchctl load ~/Library/LaunchAgents/com.jimmyg.llama-server.plist
```

## Hermes Agent Integration

Two custom providers in `~/.hermes/config.yaml`:

```yaml
custom_providers:
  - base_url: http://localhost:8085/v1
    model: /Users/jimmyg/models/Qwen3.6-35B-A3B-Q8_0.gguf
    name: qwen35b
  - base_url: http://localhost:8080/v1
    model: Qwen3.6-27B-Q8_0
    name: qwen27b
```

## Baseline Performance

At this point with default settings:
- 35B MoE: ~20 t/s
- 27B dense: ~12 t/s

Plenty of room for optimization.