2 Commits

Author SHA1 Message Date
Gan, Jimmy f7b8301824 docs: add architecture reference, update CLAUDE.md with current CI/sidebar state
Deploy Dashboard (Main) / Backend Tests (push) Failing after 2m50s
Deploy Dashboard (Main) / Frontend Tests (push) Successful in 10s
Deploy Dashboard (Main) / Build Main Image (push) Successful in 29m11s
Deploy Dashboard (Main) / Deploy to Production (push) Successful in 1m0s
2026-07-09 23:37:13 +08:00
Gan, Jimmy e66dc9a38a fix(tests): remove Docker tests + RBAC fixtures (routes removed)
Deploy Dashboard (Dev) / Backend Tests (push) Failing after 1m6s
Deploy Dashboard (Dev) / Build Dev Image (push) Has been skipped
Deploy Dashboard (Dev) / Frontend Tests (push) Successful in 52s
Deploy Dashboard (Dev) / Deploy to Dev (push) Has been skipped
2026-07-09 23:36:39 +08:00
4 changed files with 146 additions and 188 deletions
+43 -27
View File
@@ -3,50 +3,66 @@
## Architecture
- Synology DS224+ (x86_64, 18GB RAM), Tailscale IP: `100.78.131.124`, LAN: `192.168.31.221`
- Docker: `/volume1/@appstore/ContainerManager/usr/bin/docker`, compose dirs: `/volume1/docker/*`
- Caddy VPS (`161.33.182.109`, Tailscale: `100.109.198.126`): reverse proxy for `nas.jimmygan.com` → NAS:4000, `music.jimmygan.com` → NAS:4533
- Build VPS (server2, `158.101.140.85`, Tailscale: `100.70.115.1`): podman, fast network for pulling images
- VPS Caddy (`161.33.182.109`, Tailscale: `100.109.198.126`): reverse proxy for `*.jimmygan.com`
- Build VPS (server2, `158.101.140.85`, Tailscale: `100.70.115.1`): podman, fast network
- NAS Caddy: port 8443 fallback only (SNI broken)
## Dashboard
- Backend: Python FastAPI (`dashboard/backend/`)
- Frontend: Svelte 5 + Tailwind CSS (`dashboard/frontend/`)
- Svelte 5 runes: use `$state`, `$props`, `$bindable` (not legacy `export let`)
- Sidebar: categorized as Main (Overview, Docker, Files, Terminal), Media (Navidrome, Jellyfin, Audiobookshelf, Immich), Tools (OpenClaw, Repos, Gitea Web, Stirling PDF, Vaultwarden, Speedtest)
- New pages: create route in `src/routes/`, import in `App.svelte`, add to appropriate category in `Sidebar.svelte`
- Sidebar categories: Main, Media, Tools (see `docs/architecture.md` for full layout)
- New pages: create route in `src/routes/`, import in `App.svelte`, update `Sidebar.svelte`
## CI/CD
Two workflows in `.gitea/workflows/`:
- **deploy.yml** — main branch: tests → build → deploy to production
- **deploy-dev.yml** — dev branch: tests → build → deploy to dev
Two runners:
- **VPS runner** (`vps` label on server2) — build + test jobs (podman)
- **NAS runner** (`nas` label on DS224+) — deploy jobs (docker, host networking)
### Build flow
1. VPS builds Docker image + pushes to local registry `100.78.131.124:5501/`
2. Streams image to NAS: `podman save | gzip | ssh nasts "gunzip | docker load"`
3. NAS deploys: `docker compose up -d`
## Local Registry
- Container: `registry-mirror` on NAS, port 5501
- Standalone mode (NOT Docker Hub proxy — blocked from China)
- VPS pushes with `podman push --tls-verify=false`
## Authentication
- **Gitea API Access:** Use a dedicated `claude-ci-bot` user with a Personal Access Token (PAT).
- **Secret Management:** Store the PAT in an environment variable `GITEA_TOKEN`. Avoid hardcoding paths to secret files.
- **Project Secrets:** Keep sensitive configurations (like the PAT) in a local `.env` file that is ignored by git.
- Build images on VPS (server2) to avoid NAS HDD I/O saturation:
1. Build on VPS with Podman
2. Stream image to NAS: `podman save <image> | ssh nasts "/volume1/@appstore/ContainerManager/usr/bin/docker load"`
3. Deploy: `git push` triggers Gitea Actions CI (builds on VPS, transfers, then `docker compose up`) — do NOT manually restart
- CI workflows are in `.gitea/workflows/` in this repo
- **Gitea API Access:** Token stored in `~/.config/tea/config.yml` (user: `jimmy`)
- **Gitea Token:** `aa8c20329040c56c079d2676318c6e33f14b221d`
- **Project Secrets:** Gitea repo secrets: CI_TOKEN, CI_DB_PASSWORD, SECRET_KEY, ADMIN_PASSWORD_HASH, TRANSMISSION_USER, TRANSMISSION_PASS
## claude-dev CI Contract
- Source package lives in `claude-dev/` (`Dockerfile`, `required-tools.txt`, and smoke scripts under `claude-dev/scripts/`)
- CI-first only: update `claude-dev/*` in git, then push; do not do ad-hoc local image rebuild/redeploy for normal changes
- Required capabilities are declared in `claude-dev/required-tools.txt`; adding tools must update this file and related smoke checks
- Build/deploy gate (`.gitea/workflows/deploy-claude-dev-dev.yml`) uses NAS runner with `DOCKER_BUILDKIT=0`, then runs:
- tools smoke (`smoke-tools.sh`)
- network smoke (`smoke-network.sh`)
- auth/mount smoke (`smoke-auth.sh` with required runtime mounts)
- Deployment only happens after smoke pass, using immutable tag format `claude-dev:dev-<timestamp>-<sha>`
- Runtime compose definition is `claude-dev/docker-compose.yml`; CI syncs it to `/volume1/docker/claude-dev/docker-compose.yml`
- Runtime requirements preserved: `network_mode: host`, `/volume1/repos` mount, `/root/.claude` mount, `/root/.claude.json` mount, `/root/.gitea_token` mount
- Rollback: run workflow manually with `rollback_image` (for example the value in `/volume1/docker/claude-dev/previous-image.txt`) to redeploy a known-good immutable tag via CI
- Source in `claude-dev/` (Dockerfile, required-tools.txt, scripts/)
- CI-first: update `claude-dev/*` in git, then push
- Workflow: `.gitea/workflows/deploy-claude-dev-dev.yml`
- Uses NAS runner with `DOCKER_BUILDKIT=0`
- Immutable tag format: `claude-dev:dev-<timestamp>-<sha>`
- Runtime compose: `claude-dev/docker-compose.yml`
- Rollback: run workflow with `rollback_image` using value from `/volume1/docker/claude-dev/previous-image.txt`
## Synology Quirks
- No `crontab` — edit `/etc/crontab` with `sudo` + `printf`, restart crond
- No `scp` subsystem — use `cat file | ssh ... 'cat > dest'`
- ACLs override Unix perms — use `sudo chmod 777` + `synoacltool -enforce-inherit` for bind mounts
- ACLs override Unix perms — use `sudo chmod 777` + `synoacltool -enforce-inherit`
- Tailscale userspace mode — cannot bind Docker ports to Tailscale IPs
## DNS
- dnsmasq on NAS resolves `*.jimmygan.com``161.33.182.109` (Caddy VPS)
- Tailscale split DNS: `jimmygan.com` → NAS (queries hit dnsmasq)
- All paths (LAN, Tailscale, public) route through Caddy for TLS
## Proxy API
- URL: `https://www.bytecatcode.org`
- Models: `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001` (no date-suffixed names)
- Models: `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001`
## Key Files
- `docs/architecture.md` — full architecture reference
- `docs/TESTING.md` — test patterns and conventions
- `.gitea/workflows/deploy.yml` — main CI
- `.gitea/workflows/deploy-dev.yml` — dev CI
+3 -3
View File
@@ -71,7 +71,7 @@ def temp_rbac_file(temp_volume_root):
data = {
"role_defaults": {
"admin": {"pages": "*", "sidebar_links": "*"},
"member": {"pages": ["dashboard", "docker"], "sidebar_links": "*"},
"member": {"pages": ["dashboard"], "sidebar_links": "*"},
"viewer": {"pages": ["dashboard"], "sidebar_links": ["dashboard"]},
},
"user_overrides": {},
@@ -470,11 +470,11 @@ def mock_rbac_data():
return {
"role_defaults": {
"admin": {"pages": "*", "sidebar_links": "*"},
"member": {"pages": ["dashboard", "docker", "files"], "sidebar_links": "*"},
"member": {"pages": ["dashboard", "files"], "sidebar_links": "*"},
"viewer": {"pages": ["dashboard"], "sidebar_links": ["dashboard"]},
},
"user_overrides": {
"customuser": {"pages": ["dashboard", "docker"], "sidebar_links": ["dashboard", "docker", "files"]}
"customuser": {"pages": ["dashboard"], "sidebar_links": ["dashboard", "files"]}
},
}
@@ -1,158 +0,0 @@
"""
Integration tests for Docker operations.
"""
import pytest
class TestDockerContainerList:
"""Test listing Docker containers."""
def test_list_containers_success(self, test_app, admin_headers):
"""Test listing containers successfully."""
response = test_app.get("/api/docker/containers", headers=admin_headers)
assert response.status_code == 200
data = response.json()
assert isinstance(data, list)
if len(data) > 0:
container = data[0]
assert "id" in container
assert "name" in container
assert "status" in container
def test_list_containers_without_auth(self, test_app):
"""Test listing containers without authentication."""
response = test_app.get("/api/docker/containers")
assert response.status_code == 401
class TestDockerContainerActions:
"""Test Docker container start/stop/restart actions."""
def test_start_container_success(self, test_app, admin_headers, mock_docker_client):
"""Test starting a container."""
container_id = "abc123"
response = test_app.post(f"/api/docker/containers/{container_id}/start", headers=admin_headers)
assert response.status_code == 200
data = response.json()
assert data["ok"] is True
def test_stop_container_success(self, test_app, admin_headers, mock_docker_client):
"""Test stopping a container."""
container_id = "abc123"
response = test_app.post(f"/api/docker/containers/{container_id}/stop", headers=admin_headers)
assert response.status_code == 200
def test_restart_container_success(self, test_app, admin_headers, mock_docker_client):
"""Test restarting a container."""
container_id = "abc123"
response = test_app.post(f"/api/docker/containers/{container_id}/restart", headers=admin_headers)
assert response.status_code == 200
def test_container_action_without_auth(self, test_app):
"""Test container action without authentication."""
response = test_app.post("/api/docker/containers/abc123/start")
assert response.status_code == 401
def test_container_action_invalid_action(self, test_app, admin_headers):
"""Test container action with invalid action."""
response = test_app.post("/api/docker/containers/abc123/invalid_action", headers=admin_headers)
assert response.status_code == 400
data = response.json()
assert "detail" in data
class TestDockerContainerLogs:
"""Test Docker container logs retrieval."""
def test_get_container_logs_success(self, test_app, admin_headers, mock_docker_client):
"""Test getting container logs."""
container_id = "abc123"
response = test_app.get(f"/api/docker/containers/{container_id}/logs", headers=admin_headers)
assert response.status_code == 200
data = response.json()
assert "logs" in data or isinstance(data, str)
def test_get_container_logs_with_tail(self, test_app, admin_headers, mock_docker_client):
"""Test getting container logs with tail limit."""
container_id = "abc123"
response = test_app.get(f"/api/docker/containers/{container_id}/logs?tail=100", headers=admin_headers)
assert response.status_code == 200
def test_get_container_logs_without_auth(self, test_app):
"""Test getting logs without authentication."""
response = test_app.get("/api/docker/containers/abc123/logs")
assert response.status_code == 401
def test_get_container_logs_nonexistent(self, test_app, admin_headers, mock_docker_client):
"""Test getting logs for nonexistent container."""
# Note: In this test setup, the mock always returns a container
# Testing actual error handling would require a different approach
# For now, verify the endpoint works with the mock
response = test_app.get("/api/docker/containers/nonexistent/logs", headers=admin_headers)
# With the current mock setup, this will succeed
assert response.status_code == 200
class TestDockerPermissions:
"""Test Docker operation permissions."""
@pytest.fixture
def member_token(self, mock_config, temp_auth_file, temp_rbac_file):
"""Create token for member role."""
from datetime import timedelta
from auth_service import create_access_token
return create_access_token(data={"sub": "memberuser", "role": "member"}, expires_delta=timedelta(minutes=30))
@pytest.fixture
def member_headers(self, member_token):
"""Headers with member access token."""
return {"Authorization": f"Bearer {member_token}"}
def test_member_can_list_containers(self, test_app, member_headers):
"""Test that member can list containers."""
response = test_app.get("/api/docker/containers", headers=member_headers)
# Members should be able to view containers
assert response.status_code == 200
def test_member_cannot_start_container(self, test_app, member_headers):
"""Test that member cannot start containers (admin only)."""
response = test_app.post("/api/docker/containers/abc123/start", headers=member_headers)
# Should be forbidden for non-admin
assert response.status_code == 403
def test_viewer_can_list_containers(self, test_app, mock_config, temp_auth_file, temp_rbac_file):
"""Test that viewer can list containers if they have docker page access."""
from datetime import timedelta
from auth_service import create_access_token
viewer_token = create_access_token(
data={"sub": "vieweruser", "role": "viewer"}, expires_delta=timedelta(minutes=30)
)
headers = {"Authorization": f"Bearer {viewer_token}"}
response = test_app.get("/api/docker/containers", headers=headers)
# Viewer may not have docker page access by default
assert response.status_code in [200, 403]
+100
View File
@@ -0,0 +1,100 @@
# NAS Tools — Architecture & Operations
## Overview
NAS Dashboard is a web UI for managing a Synology DS224+ NAS and its ecosystem of services. It runs as a Docker container on the NAS with a FastAPI backend + Svelte 5 frontend.
## Architecture
```
Browser → VPS Caddy (161.33.182.109, port 443)
→ Tailscale → NAS Caddy (port 8443, fallback only)
→ NAS Dashboard (port 4000)
```
DNS: `*.jimmygan.com` → VPS Caddy → Tailscale → NAS services.
## Services
### Media
| Service | URL | Port | Description |
|---------|-----|------|-------------|
| Navidrome | music.jimmygan.com | 4533 | Music streaming |
| Jellyfin | media.jimmygan.com | 8097 | Media server |
| Audiobookshelf | books.jimmygan.com | 13378 | Audiobooks & podcasts |
| Immich | photos.jimmygan.com | 2283 | Photo/video backup |
| t-youtube | yt.jimmygan.com | 8001 | YouTube subscription reader |
| Transmission | (sidebar route) | 9091 | BitTorrent client |
### Tools
| Service | URL | Description |
|---------|-----|-------------|
| Gitea | git.jimmygan.com | Git hosting + CI (Gitea Actions) |
| LiteLLM | (dashboard route) | LLM proxy |
| OPC | (dashboard route) | Kanban + agents |
| n8n | n8n.jimmygan.com | Workflow automation |
| Vaultwarden | vault.jimmygan.com | Password manager |
| Hermes | hermes-agent.nousresearch.com/docs | AI Agent docs |
## CI/CD (Gitea Actions)
### Workflows
- **deploy.yml** — Main branch: tests → build → deploy to production
- **deploy-dev.yml** — Dev branch: tests → build → deploy to dev
### Runners
- **VPS runner** (`vps` label) — server2 (Oracle Tokyo, 158.101.140.85). Used for build + test jobs. Uses podman.
- **NAS runner** (`nas` label) — DS224+. Used for deploy jobs. Uses Docker via `/volume1/@appstore/ContainerManager/usr/bin/docker`. Runs with `nas:host` label (host networking).
### Build & Deploy Flow
1. VPS runner builds Docker image
2. Pushes to local registry: `100.78.131.124:5501/nas-dashboard:latest` (podman push --tls-verify=false)
3. Streams image to NAS via SSH pipe: `podman save | gzip | ssh nasts "gunzip | docker load"`
4. NAS runner deploys: `docker compose up -d`
### CI Speed Notes
- Build is the bottleneck (~12 min first run due to npm ci + Vite build)
- npm ci is cached between runs (node_modules saved to disk on VPS)
- pip is cached between runs
- Registry push enables future layer caching
- Synology Docker daemon doesn't support `--tls-verify=false` for pulls; SSH pipe used instead
### Recent CI Fixes (July 2026)
| Fix | Date | Description |
|-----|------|-------------|
| Checkout URL | Jul 8 | Changed from `gitea:3000` to `127.0.0.1:3300` (NAS host networking) |
| TRANSMISSION_USER/PASS | Jul 8 | Added missing env vars to deploy step |
| Gzip transfer | Jul 8 | Added gzip compression to SSH pipe |
| Local registry | Jul 9 | Build pushes to local registry for layer caching |
| NAS runner crash | Jul 8 | `.runner` address changed from `gitea:3000` to `127.0.0.1:3300` |
## Local Registry
Registry container: `registry-mirror` on NAS, port 5501.
- Standalone mode (NOT Docker Hub proxy — Hub is blocked from China)
- Listens on `0.0.0.0:5501`
- Images pushed by VPS CI, pulled by NAS fallback via SSH pipe
- API: `http://100.78.131.124:5501/v2/`
## Dashboard Sidebar Layout
### Main
Overview, LiteLLM, OPC, Files, Terminal, Security
### Media
Navidrome, Jellyfin, Audiobookshelf, Immich, t-youtube, Media Management, Transmission
### Tools
Hermes, Chat Digest, Code Sessions, Repos, Gitea Web, Stirling PDF, Vaultwarden, n8n, Speedtest
All external links use clean HTTPS URLs (no `:8443` port).
## Cmd+K Palette
Press `⌘K` (Mac) or `Ctrl+K` to fuzzy-search all sidebar items. New in July 2026.
## NAS Quirks
- No `crontab` — edit `/etc/crontab` with `sudo` + `printf`, restart crond
- No `scp` subsystem — use `cat file | ssh ... 'cat > dest'`
- ACLs override Unix perms — use `sudo chmod 777` + `synoacltool -enforce-inherit`
- Docker at `/volume1/@appstore/ContainerManager/usr/bin/docker`
- Tailscale userspace mode — cannot bind to Tailscale IP directly in Docker