6 Commits

Author SHA1 Message Date
Gan, Jimmy 5b622e8232 chore: sync main into dev 2026-07-10 13:15:10 +08:00
Gan, Jimmy 4203bef3d4 fix(tests): remove stale 'docker' and 'openclaw' routes from DEFAULT_RBAC member pages
Deploy Dashboard (Main) / Backend Tests (push) Successful in 3m35s
Deploy Dashboard (Main) / Frontend Tests (push) Successful in 5s
Deploy Dashboard (Main) / Build Main Image (push) Successful in 1m44s
Deploy Dashboard (Main) / Deploy to Production (push) Successful in 1m15s
2026-07-10 12:51:52 +08:00
Gan, Jimmy f549675630 fix(tests): remove stale 'docker' and 'openclaw' routes from DEFAULT_RBAC member pages
Deploy Dashboard (Dev) / Backend Tests (push) Successful in 2m14s
Deploy Dashboard (Dev) / Frontend Tests (push) Successful in 1m30s
Deploy Dashboard (Dev) / Build Dev Image (push) Successful in 1m36s
Deploy Dashboard (Dev) / Deploy to Dev (push) Successful in 52s
2026-07-10 12:51:41 +08:00
Gan, Jimmy b3a78aa166 add PDD workflow: CLAUDE.md, PDD.md, active.md symlink 2026-07-09 23:57:06 +08:00
Gan, Jimmy f7b8301824 docs: add architecture reference, update CLAUDE.md with current CI/sidebar state
Deploy Dashboard (Main) / Backend Tests (push) Failing after 2m50s
Deploy Dashboard (Main) / Frontend Tests (push) Successful in 10s
Deploy Dashboard (Main) / Build Main Image (push) Successful in 29m11s
Deploy Dashboard (Main) / Deploy to Production (push) Successful in 1m0s
2026-07-09 23:37:13 +08:00
Gan, Jimmy e66dc9a38a fix(tests): remove Docker tests + RBAC fixtures (routes removed)
Deploy Dashboard (Dev) / Backend Tests (push) Failing after 1m6s
Deploy Dashboard (Dev) / Build Dev Image (push) Has been skipped
Deploy Dashboard (Dev) / Frontend Tests (push) Successful in 52s
Deploy Dashboard (Dev) / Deploy to Dev (push) Has been skipped
2026-07-09 23:36:39 +08:00
8 changed files with 176 additions and 199 deletions
+48 -36
View File
@@ -1,52 +1,64 @@
# NAS Tools Project # NAS Tools Project
## Architecture ## PDD Workflow
- Synology DS224+ (x86_64, 18GB RAM), Tailscale IP: `100.78.131.124`, LAN: `192.168.31.221`
- Docker: `/volume1/@appstore/ContainerManager/usr/bin/docker`, compose dirs: `/volume1/docker/*`
- Caddy VPS (`161.33.182.109`, Tailscale: `100.109.198.126`): reverse proxy for `nas.jimmygan.com` → NAS:4000, `music.jimmygan.com` → NAS:4533
- Build VPS (server2, `158.101.140.85`, Tailscale: `100.70.115.1`): podman, fast network for pulling images
## Dashboard This project follows **PDD (Product Definition Document)** development.
**Always read these files before writing code:**
- `docs/PDD.md` — product vision, goals, targets (the north star)
- `docs/specs/active.md` — current sprint spec (what we're building now)
**Workflow:**
1. PDD → Sprint Spec → Implementation → Validate ACs
2. Scope changes update the sprint spec first, code second
3. Never ship without ACs passing
## Dev Commands
- **Build dashboard:** `cd dashboard/frontend && npm run build`
- **Dev dashboard:** `cd dashboard/backend && uvicorn main:app --reload` + `cd dashboard/frontend && npm run dev`
- **Test:** `cd dashboard && python -m pytest backend/tests/`
- **Deploy:** Push to main → Gitea CI builds + deploys to NAS
## Architecture
- **Synology DS224+** (x86_64, 18GB RAM), Tailscale IP: `100.78.131.124`, LAN: `192.168.31.221`
- **Docker:** `/volume1/@appstore/ContainerManager/usr/bin/docker`, compose dirs: `/volume1/docker/*`
- **VPS Caddy** (`161.33.182.109`, Tailscale: `100.109.198.126`): reverse proxy for `*.jimmygan.com`
- **Build VPS** (server2, `158.101.140.85`, Tailscale: `100.70.115.1`): podman builds
- **NAS Caddy:** port 8443 fallback only (SNI broken)
### Dashboard
- Backend: Python FastAPI (`dashboard/backend/`) - Backend: Python FastAPI (`dashboard/backend/`)
- Frontend: Svelte 5 + Tailwind CSS (`dashboard/frontend/`) - Frontend: Svelte 5 + Tailwind CSS (`dashboard/frontend/`)
- Svelte 5 runes: use `$state`, `$props`, `$bindable` (not legacy `export let`) - Svelte 5 runes: `$state`, `$props`, `$bindable` (not `export let`)
- Sidebar: categorized as Main (Overview, Docker, Files, Terminal), Media (Navidrome, Jellyfin, Audiobookshelf, Immich), Tools (OpenClaw, Repos, Gitea Web, Stirling PDF, Vaultwarden, Speedtest) - New pages: create route in `src/routes/`, import in `App.svelte`, update `Sidebar.svelte`
- New pages: create route in `src/routes/`, import in `App.svelte`, add to appropriate category in `Sidebar.svelte`
## Authentication ### CI/CD
- **Gitea API Access:** Use a dedicated `claude-ci-bot` user with a Personal Access Token (PAT). Two Gitea workflows in `.gitea/workflows/`:
- **Secret Management:** Store the PAT in an environment variable `GITEA_TOKEN`. Avoid hardcoding paths to secret files. - **deploy.yml** — main branch: tests → build → deploy to production
- **Project Secrets:** Keep sensitive configurations (like the PAT) in a local `.env` file that is ignored by git. - **deploy-dev.yml** — dev branch: tests → build → deploy to dev
- Build images on VPS (server2) to avoid NAS HDD I/O saturation: Build flow: VPS builds → pushes to local registry (`100.78.131.124:5501/`) → streams to NAS → `docker compose up -d`
1. Build on VPS with Podman
2. Stream image to NAS: `podman save <image> | ssh nasts "/volume1/@appstore/ContainerManager/usr/bin/docker load"`
3. Deploy: `git push` triggers Gitea Actions CI (builds on VPS, transfers, then `docker compose up`) — do NOT manually restart
- CI workflows are in `.gitea/workflows/` in this repo
## claude-dev CI Contract ## Key Files
- Source package lives in `claude-dev/` (`Dockerfile`, `required-tools.txt`, and smoke scripts under `claude-dev/scripts/`)
- CI-first only: update `claude-dev/*` in git, then push; do not do ad-hoc local image rebuild/redeploy for normal changes - `docs/architecture.md` — full architecture reference
- Required capabilities are declared in `claude-dev/required-tools.txt`; adding tools must update this file and related smoke checks - `docs/TESTING.md` — test patterns and conventions
- Build/deploy gate (`.gitea/workflows/deploy-claude-dev-dev.yml`) uses NAS runner with `DOCKER_BUILDKIT=0`, then runs: - `specs/` — sprint history
- tools smoke (`smoke-tools.sh`) - `.gitea/workflows/deploy.yml` — main CI
- network smoke (`smoke-network.sh`) - `.gitea/workflows/deploy-dev.yml` — dev CI
- auth/mount smoke (`smoke-auth.sh` with required runtime mounts) - `dashboard/backend/main.py` — FastAPI entry point
- Deployment only happens after smoke pass, using immutable tag format `claude-dev:dev-<timestamp>-<sha>` - `dashboard/frontend/src/App.svelte` — Svelte entry point
- Runtime compose definition is `claude-dev/docker-compose.yml`; CI syncs it to `/volume1/docker/claude-dev/docker-compose.yml`
- Runtime requirements preserved: `network_mode: host`, `/volume1/repos` mount, `/root/.claude` mount, `/root/.claude.json` mount, `/root/.gitea_token` mount
- Rollback: run workflow manually with `rollback_image` (for example the value in `/volume1/docker/claude-dev/previous-image.txt`) to redeploy a known-good immutable tag via CI
## Synology Quirks ## Synology Quirks
- No `crontab` — edit `/etc/crontab` with `sudo` + `printf`, restart crond - No `crontab` — edit `/etc/crontab` with `sudo` + `printf`, restart crond
- No `scp` subsystem — use `cat file | ssh ... 'cat > dest'` - No `scp` subsystem — use `cat file | ssh ... 'cat > dest'`
- ACLs override Unix perms — use `sudo chmod 777` + `synoacltool -enforce-inherit` for bind mounts - ACLs override Unix perms — use `sudo chmod 777` + `synoacltool -enforce-inherit`
- Tailscale userspace mode — cannot bind Docker ports to Tailscale IPs
## DNS ## DNS
- dnsmasq on NAS resolves `*.jimmygan.com``161.33.182.109` (Caddy VPS) - dnsmasq on NAS resolves `*.jimmygan.com``161.33.182.109` (Caddy VPS)
- Tailscale split DNS: `jimmygan.com` → NAS (queries hit dnsmasq) - Tailscale split DNS: `jimmygan.com` → NAS (queries hit dnsmasq)
- All paths (LAN, Tailscale, public) route through Caddy for TLS
## Proxy API
- URL: `https://www.bytecatcode.org`
- Models: `claude-opus-4-6`, `claude-sonnet-4-6`, `claude-haiku-4-5-20251001` (no date-suffixed names)
+1 -1
View File
@@ -26,7 +26,7 @@ ROLE_GROUP_MAP = {
DEFAULT_RBAC = { DEFAULT_RBAC = {
"role_defaults": { "role_defaults": {
"admin": {"pages": "*", "sidebar_links": "*"}, "admin": {"pages": "*", "sidebar_links": "*"},
"member": {"pages": ["dashboard", "docker", "files", "openclaw", "chat-digest", "opc"], "sidebar_links": "*"}, "member": {"pages": ["dashboard", "files", "chat-digest", "opc"], "sidebar_links": "*"},
"viewer": {"pages": ["dashboard"], "sidebar_links": ["dashboard"]}, "viewer": {"pages": ["dashboard"], "sidebar_links": ["dashboard"]},
}, },
"user_overrides": {}, "user_overrides": {},
+3 -3
View File
@@ -71,7 +71,7 @@ def temp_rbac_file(temp_volume_root):
data = { data = {
"role_defaults": { "role_defaults": {
"admin": {"pages": "*", "sidebar_links": "*"}, "admin": {"pages": "*", "sidebar_links": "*"},
"member": {"pages": ["dashboard", "docker"], "sidebar_links": "*"}, "member": {"pages": ["dashboard"], "sidebar_links": "*"},
"viewer": {"pages": ["dashboard"], "sidebar_links": ["dashboard"]}, "viewer": {"pages": ["dashboard"], "sidebar_links": ["dashboard"]},
}, },
"user_overrides": {}, "user_overrides": {},
@@ -470,11 +470,11 @@ def mock_rbac_data():
return { return {
"role_defaults": { "role_defaults": {
"admin": {"pages": "*", "sidebar_links": "*"}, "admin": {"pages": "*", "sidebar_links": "*"},
"member": {"pages": ["dashboard", "docker", "files"], "sidebar_links": "*"}, "member": {"pages": ["dashboard", "files"], "sidebar_links": "*"},
"viewer": {"pages": ["dashboard"], "sidebar_links": ["dashboard"]}, "viewer": {"pages": ["dashboard"], "sidebar_links": ["dashboard"]},
}, },
"user_overrides": { "user_overrides": {
"customuser": {"pages": ["dashboard", "docker"], "sidebar_links": ["dashboard", "docker", "files"]} "customuser": {"pages": ["dashboard"], "sidebar_links": ["dashboard", "files"]}
}, },
} }
@@ -1,158 +0,0 @@
"""
Integration tests for Docker operations.
"""
import pytest
class TestDockerContainerList:
"""Test listing Docker containers."""
def test_list_containers_success(self, test_app, admin_headers):
"""Test listing containers successfully."""
response = test_app.get("/api/docker/containers", headers=admin_headers)
assert response.status_code == 200
data = response.json()
assert isinstance(data, list)
if len(data) > 0:
container = data[0]
assert "id" in container
assert "name" in container
assert "status" in container
def test_list_containers_without_auth(self, test_app):
"""Test listing containers without authentication."""
response = test_app.get("/api/docker/containers")
assert response.status_code == 401
class TestDockerContainerActions:
"""Test Docker container start/stop/restart actions."""
def test_start_container_success(self, test_app, admin_headers, mock_docker_client):
"""Test starting a container."""
container_id = "abc123"
response = test_app.post(f"/api/docker/containers/{container_id}/start", headers=admin_headers)
assert response.status_code == 200
data = response.json()
assert data["ok"] is True
def test_stop_container_success(self, test_app, admin_headers, mock_docker_client):
"""Test stopping a container."""
container_id = "abc123"
response = test_app.post(f"/api/docker/containers/{container_id}/stop", headers=admin_headers)
assert response.status_code == 200
def test_restart_container_success(self, test_app, admin_headers, mock_docker_client):
"""Test restarting a container."""
container_id = "abc123"
response = test_app.post(f"/api/docker/containers/{container_id}/restart", headers=admin_headers)
assert response.status_code == 200
def test_container_action_without_auth(self, test_app):
"""Test container action without authentication."""
response = test_app.post("/api/docker/containers/abc123/start")
assert response.status_code == 401
def test_container_action_invalid_action(self, test_app, admin_headers):
"""Test container action with invalid action."""
response = test_app.post("/api/docker/containers/abc123/invalid_action", headers=admin_headers)
assert response.status_code == 400
data = response.json()
assert "detail" in data
class TestDockerContainerLogs:
"""Test Docker container logs retrieval."""
def test_get_container_logs_success(self, test_app, admin_headers, mock_docker_client):
"""Test getting container logs."""
container_id = "abc123"
response = test_app.get(f"/api/docker/containers/{container_id}/logs", headers=admin_headers)
assert response.status_code == 200
data = response.json()
assert "logs" in data or isinstance(data, str)
def test_get_container_logs_with_tail(self, test_app, admin_headers, mock_docker_client):
"""Test getting container logs with tail limit."""
container_id = "abc123"
response = test_app.get(f"/api/docker/containers/{container_id}/logs?tail=100", headers=admin_headers)
assert response.status_code == 200
def test_get_container_logs_without_auth(self, test_app):
"""Test getting logs without authentication."""
response = test_app.get("/api/docker/containers/abc123/logs")
assert response.status_code == 401
def test_get_container_logs_nonexistent(self, test_app, admin_headers, mock_docker_client):
"""Test getting logs for nonexistent container."""
# Note: In this test setup, the mock always returns a container
# Testing actual error handling would require a different approach
# For now, verify the endpoint works with the mock
response = test_app.get("/api/docker/containers/nonexistent/logs", headers=admin_headers)
# With the current mock setup, this will succeed
assert response.status_code == 200
class TestDockerPermissions:
"""Test Docker operation permissions."""
@pytest.fixture
def member_token(self, mock_config, temp_auth_file, temp_rbac_file):
"""Create token for member role."""
from datetime import timedelta
from auth_service import create_access_token
return create_access_token(data={"sub": "memberuser", "role": "member"}, expires_delta=timedelta(minutes=30))
@pytest.fixture
def member_headers(self, member_token):
"""Headers with member access token."""
return {"Authorization": f"Bearer {member_token}"}
def test_member_can_list_containers(self, test_app, member_headers):
"""Test that member can list containers."""
response = test_app.get("/api/docker/containers", headers=member_headers)
# Members should be able to view containers
assert response.status_code == 200
def test_member_cannot_start_container(self, test_app, member_headers):
"""Test that member cannot start containers (admin only)."""
response = test_app.post("/api/docker/containers/abc123/start", headers=member_headers)
# Should be forbidden for non-admin
assert response.status_code == 403
def test_viewer_can_list_containers(self, test_app, mock_config, temp_auth_file, temp_rbac_file):
"""Test that viewer can list containers if they have docker page access."""
from datetime import timedelta
from auth_service import create_access_token
viewer_token = create_access_token(
data={"sub": "vieweruser", "role": "viewer"}, expires_delta=timedelta(minutes=30)
)
headers = {"Authorization": f"Bearer {viewer_token}"}
response = test_app.get("/api/docker/containers", headers=headers)
# Viewer may not have docker page access by default
assert response.status_code in [200, 403]
@@ -182,7 +182,6 @@ class TestGetPages:
pages = get_pages("testuser", "member") pages = get_pages("testuser", "member")
assert isinstance(pages, list) assert isinstance(pages, list)
assert "dashboard" in pages assert "dashboard" in pages
assert "docker" in pages
def test_get_pages_viewer_default(self, mock_config, temp_rbac_file): def test_get_pages_viewer_default(self, mock_config, temp_rbac_file):
"""Test getting pages for viewer with default settings.""" """Test getting pages for viewer with default settings."""
+23
View File
@@ -0,0 +1,23 @@
# Product Definition Document — NAS Tools
## Vision
A single dashboard to manage all self-hosted services on the NAS — Docker containers, media, tools, system health — accessible from anywhere via a clean web UI.
## Goals
1. **Unified management** — one dashboard for all NAS services (containers, media, tools, auth)
2. **GFW-proof deployment** — all traffic routes through VPS + Tailscale, works from China
3. **Self-healing** — auto-detect down containers, email alerts, health checks
4. **CI/CD from Gitea** — push to deploy with zero manual SSH
## Target Users
Self (Jimmy Gan). Secondary: family members who access media services on the NAS.
## Success Metrics
- Dashboard loads in < 3s from anywhere
- All CI workflows green
- Zero manual SSH deploys
- < 5min recovery from container crash
+100
View File
@@ -0,0 +1,100 @@
# NAS Tools — Architecture & Operations
## Overview
NAS Dashboard is a web UI for managing a Synology DS224+ NAS and its ecosystem of services. It runs as a Docker container on the NAS with a FastAPI backend + Svelte 5 frontend.
## Architecture
```
Browser → VPS Caddy (161.33.182.109, port 443)
→ Tailscale → NAS Caddy (port 8443, fallback only)
→ NAS Dashboard (port 4000)
```
DNS: `*.jimmygan.com` → VPS Caddy → Tailscale → NAS services.
## Services
### Media
| Service | URL | Port | Description |
|---------|-----|------|-------------|
| Navidrome | music.jimmygan.com | 4533 | Music streaming |
| Jellyfin | media.jimmygan.com | 8097 | Media server |
| Audiobookshelf | books.jimmygan.com | 13378 | Audiobooks & podcasts |
| Immich | photos.jimmygan.com | 2283 | Photo/video backup |
| t-youtube | yt.jimmygan.com | 8001 | YouTube subscription reader |
| Transmission | (sidebar route) | 9091 | BitTorrent client |
### Tools
| Service | URL | Description |
|---------|-----|-------------|
| Gitea | git.jimmygan.com | Git hosting + CI (Gitea Actions) |
| LiteLLM | (dashboard route) | LLM proxy |
| OPC | (dashboard route) | Kanban + agents |
| n8n | n8n.jimmygan.com | Workflow automation |
| Vaultwarden | vault.jimmygan.com | Password manager |
| Hermes | hermes-agent.nousresearch.com/docs | AI Agent docs |
## CI/CD (Gitea Actions)
### Workflows
- **deploy.yml** — Main branch: tests → build → deploy to production
- **deploy-dev.yml** — Dev branch: tests → build → deploy to dev
### Runners
- **VPS runner** (`vps` label) — server2 (Oracle Tokyo, 158.101.140.85). Used for build + test jobs. Uses podman.
- **NAS runner** (`nas` label) — DS224+. Used for deploy jobs. Uses Docker via `/volume1/@appstore/ContainerManager/usr/bin/docker`. Runs with `nas:host` label (host networking).
### Build & Deploy Flow
1. VPS runner builds Docker image
2. Pushes to local registry: `100.78.131.124:5501/nas-dashboard:latest` (podman push --tls-verify=false)
3. Streams image to NAS via SSH pipe: `podman save | gzip | ssh nasts "gunzip | docker load"`
4. NAS runner deploys: `docker compose up -d`
### CI Speed Notes
- Build is the bottleneck (~12 min first run due to npm ci + Vite build)
- npm ci is cached between runs (node_modules saved to disk on VPS)
- pip is cached between runs
- Registry push enables future layer caching
- Synology Docker daemon doesn't support `--tls-verify=false` for pulls; SSH pipe used instead
### Recent CI Fixes (July 2026)
| Fix | Date | Description |
|-----|------|-------------|
| Checkout URL | Jul 8 | Changed from `gitea:3000` to `127.0.0.1:3300` (NAS host networking) |
| TRANSMISSION_USER/PASS | Jul 8 | Added missing env vars to deploy step |
| Gzip transfer | Jul 8 | Added gzip compression to SSH pipe |
| Local registry | Jul 9 | Build pushes to local registry for layer caching |
| NAS runner crash | Jul 8 | `.runner` address changed from `gitea:3000` to `127.0.0.1:3300` |
## Local Registry
Registry container: `registry-mirror` on NAS, port 5501.
- Standalone mode (NOT Docker Hub proxy — Hub is blocked from China)
- Listens on `0.0.0.0:5501`
- Images pushed by VPS CI, pulled by NAS fallback via SSH pipe
- API: `http://100.78.131.124:5501/v2/`
## Dashboard Sidebar Layout
### Main
Overview, LiteLLM, OPC, Files, Terminal, Security
### Media
Navidrome, Jellyfin, Audiobookshelf, Immich, t-youtube, Media Management, Transmission
### Tools
Hermes, Chat Digest, Code Sessions, Repos, Gitea Web, Stirling PDF, Vaultwarden, n8n, Speedtest
All external links use clean HTTPS URLs (no `:8443` port).
## Cmd+K Palette
Press `⌘K` (Mac) or `Ctrl+K` to fuzzy-search all sidebar items. New in July 2026.
## NAS Quirks
- No `crontab` — edit `/etc/crontab` with `sudo` + `printf`, restart crond
- No `scp` subsystem — use `cat file | ssh ... 'cat > dest'`
- ACLs override Unix perms — use `sudo chmod 777` + `synoacltool -enforce-inherit`
- Docker at `/volume1/@appstore/ContainerManager/usr/bin/docker`
- Tailscale userspace mode — cannot bind to Tailscale IP directly in Docker
+1
View File
@@ -0,0 +1 @@
../../specs/sprint-06-code-quality.md