fix(ci): remove --cache-from flag - Podman caches layers automatically

feat(ci): build dashboard images on VPS (SSD) instead of NAS HDD
- Add build-image job (runs-on: vps) for deploy.yml (main) - builds with Podman on server2, pipes finished image to NAS via SSH, retags - Add build-image-dev job (runs-on: vps) for deploy-dev.yml (dev) - same pattern for dev images - Remove build + base image pre-warm from NAS-side deploy jobs (image is now pre-loaded by the build job) - deploy job condition: always() && build-image succeeded (so frontend test failure doesn't block deployment, but build failure does) NAS HDD I/O was the bottleneck. VPS has SSD and same x86_64 arch.
2026-06-04 17:52:26 +08:00 · 2026-06-04 17:49:43 +08:00 · 2026-06-04 11:42:16 +08:00 · 2026-06-04 11:22:41 +08:00
2 changed files with 72 additions and 119 deletions
@@ -157,10 +157,41 @@ jobs:
          npm run test:coverage -- --reporter=verbose --run --pool=forks --poolOptions.forks.maxForks=2 || true
          echo "Frontend tests completed (pre-existing Vite compatibility issues may cause failures)"

+  build-image-dev:
+    name: Build Dev Image
+    runs-on: vps
+    needs: [backend-tests]
+
+    steps:
+      - name: Checkout repository
+        run: |
+          if [ ! -d .git ]; then
+            git clone --depth=1 "http://100.78.131.124:3300/${GITHUB_REPOSITORY}.git" .
+          fi
+
+      - name: Build dev Docker image
+        run: |
+          set -e
+          echo "Building on VPS (SSD)..."
+          CACHE_ARG=""
+          if podman image exists nas-dashboard-dev:latest 2>/dev/null; then
+            CACHE_ARG="--cache-from nas-dashboard-dev:latest"
+          fi
+          podman build $CACHE_ARG -t nas-dashboard-dev:latest dashboard/
+
+      - name: Transfer image to NAS
+        run: |
+          set -e
+          echo "Streaming dev image to NAS..."
+          podman save nas-dashboard-dev:latest | ssh nasts "/volume1/@appstore/ContainerManager/usr/bin/docker load 2>&1"
+          echo "Retagging image..."
+          ssh nasts "/volume1/@appstore/ContainerManager/usr/bin/docker tag localhost/nas-dashboard-dev:latest nas-dashboard-dev:latest 2>&1 && /volume1/@appstore/ContainerManager/usr/bin/docker image rm localhost/nas-dashboard-dev:latest 2>&1"
+          echo "Dev image transferred and retagged successfully"
+
  deploy-dev:
    name: Deploy to Dev
    runs-on: ubuntu-latest
-    needs: [backend-tests, frontend-tests]
+    needs: [build-image-dev, frontend-tests]
    env:
      SECRET_KEY: test-secret-key-for-ci-environment-32chars-minimum
    container:
@@ -175,42 +206,6 @@ jobs:
            git clone --depth=1 "http://gitea:3000/${GITHUB_REPOSITORY}.git" .
          fi

-      - name: Warm mirror cache for base images
-        run: |
-          set -u
-          mirror_host="100.78.131.124:5501"
-
-          prewarm_base_image() {
-            image_ref="$1"
-            if docker image inspect "${image_ref}" >/dev/null 2>&1; then
-              echo "Image ${image_ref} already exists locally, skipping pre-pull"
-              return 0
-            fi
-
-            mirror_ref="${mirror_host}/library/${image_ref}"
-            echo "Attempting mirror pre-pull: ${mirror_ref}"
-            if timeout 30 docker pull "${mirror_ref}" 2>/dev/null; then
-              docker tag "${mirror_ref}" "${image_ref}"
-              echo "Mirror pre-pull succeeded for ${image_ref}"
-            else
-              echo "Mirror pre-pull failed for ${image_ref}, will pull during build"
-            fi
-          }
-
-          prewarm_base_image "node:20-alpine"
-          prewarm_base_image "python:3.12-slim"
-
-      - name: Build dev image
-        run: |
-          set -e
-          export DOCKER_API_VERSION=1.43
-          if ! DOCKER_BUILDKIT=0 docker build --cache-from nas-dashboard-dev:latest -t nas-dashboard-dev:latest dashboard/; then
-            echo "Build failed. Checking for network issues..."
-            curl -I https://registry.npmmirror.com || echo "npmmirror unreachable"
-            curl -I https://pypi.tuna.tsinghua.edu.cn || echo "Tsinghua PyPI unreachable"
-            exit 1
-          fi
-
      - name: Sync runtime compose file
        run: |
          mkdir -p /volume1/docker/nas-dashboard
@@ -14,7 +14,7 @@ concurrency:
 jobs:
  backend-tests:
    name: Backend Tests
-    runs-on: vps
+    runs-on: ubuntu-latest
    env:
      SECRET_KEY: test-secret-key-for-ci-environment-32chars-minimum

@@ -22,7 +22,7 @@ jobs:
      - name: Checkout repository
        run: |
          if [ ! -d .git ]; then
-            git clone --depth=1 "http://100.78.131.124:3300/${GITHUB_REPOSITORY}.git" .
+            git clone --depth=1 "http://172.21.0.1:3300/${GITHUB_REPOSITORY}.git" .
          fi

      - name: Setup Python
@@ -168,17 +168,44 @@ jobs:
          fi
          echo "✅ Frontend tests passed"

+  build-image:
+    name: Build Docker Image
+    runs-on: vps
+    needs: [backend-tests]
+
+    steps:
+      - name: Checkout repository
+        run: |
+          if [ ! -d .git ]; then
+            git clone --depth=1 "http://100.78.131.124:3300/${GITHUB_REPOSITORY}.git" .
+          fi
+
+      - name: Build Docker image
+        run: |
+          set -e
+          echo "Building on VPS (SSD)..."
+          # Podman caches layers automatically; no --cache-from needed
+          podman build -t nas-dashboard:latest dashboard/
+
+      - name: Transfer image to NAS
+        run: |
+          set -e
+          echo "Streaming image to NAS..."
+          podman save nas-dashboard:latest | ssh nasts "/volume1/@appstore/ContainerManager/usr/bin/docker load 2>&1"
+          echo "Retagging image..."
+          ssh nasts "/volume1/@appstore/ContainerManager/usr/bin/docker tag localhost/nas-dashboard:latest nas-dashboard:latest 2>&1 && /volume1/@appstore/ContainerManager/usr/bin/docker image rm localhost/nas-dashboard:latest 2>&1"
+          echo "Image transferred and retagged successfully"
+
  deploy:
    name: Deploy to Production
    runs-on: nas
-    if: always()
-    needs: [backend-tests, frontend-tests]
+    if: always() && needs.build-image.result == 'success'
+    needs: [build-image, frontend-tests]
    env:
      SECRET_KEY: test-secret-key-for-ci-environment-32chars-minimum
      GITEA_DB_PASSWORD: nyM3P4v1aYAsT7zfUtdguImNVHgFltCKCY/OjzWZVRE=
    container:
      network: gitea_gitea
-      options: --add-host=host.docker.internal:host-gateway --dns 192.168.31.1 --dns 8.8.8.8
      volumes:
        - /var/packages/ContainerManager/target/usr/bin/docker:/usr/bin/docker
        - /volume1/docker/nas-dashboard:/nas-dashboard
@@ -189,58 +216,6 @@ jobs:
            git clone --depth=1 "http://gitea:3000/${GITHUB_REPOSITORY}.git" .
          fi

-      - name: Warm mirror cache for base images
-        run: |
-          set -u
-          mirror_host="100.78.131.124:5501"
-
-          if command -v curl >/dev/null 2>&1; then
-            if curl -fsS --max-time 10 "http://${mirror_host}/v2/" >/dev/null; then
-              echo "Registry mirror is healthy at ${mirror_host}"
-            else
-              echo "Warning: registry mirror health check failed at ${mirror_host}; continuing with normal pull path"
-            fi
-          else
-            echo "Warning: curl is unavailable; skipping explicit mirror health check"
-          fi
-
-          prewarm_base_image() {
-            image_ref="$1"
-            # Skip if image already exists locally
-            if docker image inspect "${image_ref}" >/dev/null 2>&1; then
-              echo "Image ${image_ref} already exists locally, skipping pre-pull"
-              return 0
-            fi
-
-            mirror_ref="${mirror_host}/library/${image_ref}"
-            echo "Attempting mirror pre-pull: ${mirror_ref}"
-            if timeout 120 docker pull "${mirror_ref}"; then
-              docker tag "${mirror_ref}" "${image_ref}"
-              echo "Mirror pre-pull succeeded for ${image_ref}"
-            else
-              echo "Warning: mirror pre-pull failed for ${image_ref}; continuing with normal pull during build"
-            fi
-          }
-
-          prewarm_base_image "node:20-alpine"
-          prewarm_base_image "python:3.12-slim"
-
-      - name: Build
-        run: |
-          set -e
-          export DOCKER_API_VERSION=1.43
-          # DOCKER_BUILDKIT=0 is required for Synology ContainerManager's Docker daemon,
-          # which does not support BuildKit syntax. Remove when Synology updates to a
-          # Docker Engine version with full BuildKit support (currently 24.x, needs 23+).
-          # --cache-to type=inline is a no-op while BuildKit is disabled; it activates
-          # automatically when DOCKER_BUILDKIT=0 is removed.
-          if ! DOCKER_BUILDKIT=0 docker build --cache-from nas-dashboard:latest -t nas-dashboard:latest dashboard/; then
-            echo "Build failed. Checking for network issues..."
-            curl -I https://registry.npmmirror.com || echo "npmmirror unreachable"
-            curl -I https://pypi.tuna.tsinghua.edu.cn || echo "Tsinghua PyPI unreachable"
-            exit 1
-          fi
-
      - name: Sync runtime compose file
        run: |
          mkdir -p /nas-dashboard
@@ -263,30 +238,7 @@ jobs:
          if ! docker container inspect nas-dashboard >/dev/null 2>&1; then
            echo "Compose failed, stripping networks and retrying..."
            docker rm -f docker-socket-proxy nas-dashboard 2>/dev/null || true
-            python3 -c "
-          import os
-          lines = open('/nas-dashboard/docker-compose.yml').readlines()
-          out = []
-          skip_net = False
-          for l in lines:
-              stripped = l.strip()
-              if stripped == 'networks:' and not l[0].isspace():
-                  skip_net = True
-                  continue
-              if skip_net:
-                  if l[0].isspace():
-                      continue
-                  skip_net = False
-              if stripped == 'networks:' and l[:4].strip() == '':
-                  skip_net = True
-                  continue
-              if skip_net:
-                  if l.startswith('      '):
-                      continue
-                  skip_net = False
-              out.append(l)
-          open('/nas-dashboard/prod-ci.yml','w').writelines(out)
-          "
+            sed '/^networks:/,$d' /nas-dashboard/docker-compose.yml > /nas-dashboard/prod-ci.yml
            docker compose -f /nas-dashboard/prod-ci.yml up -d --pull never 2>&1 || true
          fi

@@ -351,7 +303,13 @@ jobs:

          # Verify network connections
          echo "=== Network connections ==="
-          docker inspect nas-dashboard --format '{{json .NetworkSettings.Networks}}' | python3 -c 'import json,sys;nets=json.load(sys.stdin);[print(f"  Connected to {n}") for n in nets];assert "nas-dashboard_internal" in nets, "Missing nas-dashboard_internal";print("All network connections established")'
+          if docker inspect nas-dashboard --format '{{json .NetworkSettings.Networks}}' | grep -q 'nas-dashboard_internal'; then
+            echo "  Connected to nas-dashboard_internal"
+            echo "All network connections established"
+          else
+            echo "❌ Missing nas-dashboard_internal network"
+            exit 1
+          fi

          # Wait for container to be healthy
          echo "Waiting for container to be healthy..."
Author	SHA1	Message	Date
Gan, Jimmy	ab44c2956e	fix(ci): remove --cache-from flag - Podman caches layers automatically Deploy Dashboard (Main) / Frontend Tests (push) Successful in 50s Details Deploy Dashboard (Main) / Backend Tests (push) Failing after 23m28s Details Deploy Dashboard (Main) / Build Docker Image (push) Has been skipped Details Deploy Dashboard (Main) / Deploy to Production (push) Failing after 14m58s Details	2026-06-04 17:52:26 +08:00
Gan, Jimmy	3c5ec3cdd2	feat(ci): build dashboard images on VPS (SSD) instead of NAS HDD Deploy Dashboard (Main) / Backend Tests (push) Has been cancelled Details Deploy Dashboard (Main) / Build Docker Image (push) Has been cancelled Details Deploy Dashboard (Main) / Deploy to Production (push) Has been cancelled Details Deploy Dashboard (Main) / Frontend Tests (push) Has been cancelled Details - Add build-image job (runs-on: vps) for deploy.yml (main) - builds with Podman on server2, pipes finished image to NAS via SSH, retags - Add build-image-dev job (runs-on: vps) for deploy-dev.yml (dev) - same pattern for dev images - Remove build + base image pre-warm from NAS-side deploy jobs (image is now pre-loaded by the build job) - deploy job condition: always() && build-image succeeded (so frontend test failure doesn't block deployment, but build failure does) NAS HDD I/O was the bottleneck. VPS has SSD and same x86_64 arch.	2026-06-04 17:49:43 +08:00
Gan, Jimmy	03b515aa1d	fix(ci): switch tests from VPS host runner to NAS ubuntu-latest Deploy Dashboard (Main) / Frontend Tests (push) Successful in 54s Details Deploy Dashboard (Main) / Backend Tests (push) Successful in 14m32s Details Deploy Dashboard (Main) / Deploy to Production (push) Failing after 15m6s Details The VPS runner's host executor has a bug where completion callbacks are silently dropped, causing jobs to appear stuck. Switched backend and frontend tests to the NAS runner which correctly reports status. Also updated checkout URL to use Docker bridge gateway 172.21.0.1:3300. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-04 11:42:16 +08:00
Gan, Jimmy	bd9cb67b14	fix(ci): replace python3 with sed/grep in deploy step containers Deploy Dashboard (Main) / Deploy to Production (push) Has been cancelled Details Deploy Dashboard (Main) / Backend Tests (push) Has been cancelled Details Deploy Dashboard (Main) / Frontend Tests (push) Has been cancelled Details The nas runner's deploy container lacks python3. Replaced: - compose networks stripping: python3 -> sed - network verification: python3 -> grep - Removed broken --dns and --add-host options from deploy container Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-04 11:22:41 +08:00