# Sprint 05 — Operations & Resilience

**Depends on:** S04 (CI must be reliable before automating ops)
**Duration:** ~12h
**Goal:** Add health checks, resource limits, log rotation, fix Immich uploads, back up dashboard data, add monitoring endpoint.

---

## S05.1 — Add HEALTHCHECK to claude-dev image

- **Files:** `claude-dev/Dockerfile`, `claude-dev/docker-compose.yml`
- **Estimate:** 1h
- **Done means:** Dockerfile has `HEALTHCHECK --interval=30s --timeout=5s --retries=3 CMD claude --version || exit 1`. `docker ps` shows healthy.
- **Verify by:**
  1. Deploy claude-dev → `docker ps` shows "(healthy)"
  2. Break `claude` binary → container shows "(unhealthy)" after 3 failures
- **Risk:** `claude --version` may require network access. Test on NAS first. Alternative: `pgrep -f claude` or simple `curl localhost:<port>`.

- [ ] S05.1 — Add HEALTHCHECK to claude-dev image

## S05.2 — Add resource limits to production containers

- **Files:** `claude-dev/docker-compose.yml`, `dashboard/docker-compose.yml`
- **Estimate:** 1.5h
- **Done means:** Production compose files have `mem_limit`, `cpus`, and `restart_policy`. Limits are generous: 2GB dashboard, 1GB claude-dev, 0.5GB sidecars.
- **Verify by:**
  1. `docker stats` shows containers respecting limits
  2. Normal operation unaffected
  3. `docker compose up -d` applies limits without restarting
- **Risk:** Limits too tight → OOM kills. Start generous, adjust after a week of monitoring.

- [ ] S05.2 — Add resource limits to production containers

## S05.3 — Implement audit log rotation

- **Files:** `dashboard/backend/main.py:161,171-188`
- **Estimate:** 2h
- **Done means:** Audit logging uses `RotatingFileHandler` with max 10MB and 5 backups.
- **Verify by:**
  1. Write 11MB of audit entries → file rotates, `audit.log.1` created
  2. Original `audit.log` starts fresh
  3. Old backups capped at 5 files
- **Risk:** Log format change may affect parsing in `system.py:82-116`. Test the log reader with rotated files.

- [ ] S05.3 — Implement audit log rotation

## S05.4 — Fix Immich ML model downloads

- **Files:** `immich/docker-compose.yml`, `immich/.env`
- **Estimate:** 3h (investigation + fix)
- **Done means:** Immich mobile upload works. ML models download successfully or ML is gracefully disabled.
- **Verify by:**
  1. Upload photo from phone → succeeds (no timeout)
  2. `ssh nas "docker logs immich_machine_learning"` → no "Failed to load detection model" errors
  3. If ML disabled: photo upload works, smart search unavailable (acceptable)
- **Risk:** Most complex item. May require pre-downloading models, configuring mirror, increasing timeouts, or disabling ML temporarily. Environment-specific (China network to modelscope.cn).

- [ ] S05.4 — Fix Immich ML model downloads

## S05.5 — Add dashboard data to backup script

- **Files:** `backup.sh`
- **Estimate:** 1.5h
- **Done means:** `backup.sh` copies `opc.db`, `auth.json`, `rbac.json`, and audit log to backup tarball. Restore procedure documented.
- **Verify by:**
  1. Run `backup.sh` → tarball contains dashboard data files
  2. Extract tarball → files are valid (SQLite DB opens, JSON parses)
- **Risk:** `auth.json` contains passkey data — ensure backup stored securely.

- [ ] S05.5 — Add dashboard data to backup script

## S05.6 — Add minimal health check endpoint

- **Files:** `dashboard/backend/main.py` or `routers/health.py`
- **Estimate:** 1h
- **Done means:** `GET /api/health` returns `{"status":"ok","db":true,"docker":true,"disk":{"free_gb":123}}`. Non-200 triggers optional Telegram notification.
- **Verify by:**
  1. All services healthy → `/api/health` returns 200
  2. Docker socket unreachable → returns 503 with `docker: false`
  3. Cron job calls `/api/health` every 5 minutes
- **Risk:** Health endpoint must be lightweight. Cache results for 30 seconds.

- [ ] S05.6 — Add minimal health check endpoint

---

## Exit criteria

- [ ] `docker ps` shows claude-dev as "(healthy)"
- [ ] Production containers have CPU/memory limits in compose
- [ ] Audit log rotates at 10MB with 5 backups
- [ ] Photo upload from phone succeeds
- [ ] `backup.sh` includes dashboard data files
- [ ] `GET /api/health` returns component statuses
- [ ] All backend tests pass
- [ ] All frontend tests pass