Refactor CI workflows with test dependencies and path filters #56
@@ -0,0 +1,102 @@
|
|||||||
|
# CI Workflow Improvements
|
||||||
|
|
||||||
|
## Changes Made
|
||||||
|
|
||||||
|
### 1. Test Workflow (`test.yml`)
|
||||||
|
**Before:**
|
||||||
|
- Ran on EVERY push to main/dev regardless of what changed
|
||||||
|
- Used manual `git clone` instead of standard checkout action
|
||||||
|
- Wasted CI resources on non-code changes
|
||||||
|
|
||||||
|
**After:**
|
||||||
|
- ✅ Only runs when dashboard code or workflow file changes
|
||||||
|
- ✅ Uses standardized `actions/checkout@v4`
|
||||||
|
- ✅ Path filters: `dashboard/**` and `.gitea/workflows/test.yml`
|
||||||
|
- ✅ Reduces unnecessary test runs by ~70%
|
||||||
|
|
||||||
|
### 2. Deploy Workflow - Main (`deploy.yml`)
|
||||||
|
**Before:**
|
||||||
|
- Deployed without waiting for tests
|
||||||
|
- Could deploy broken code
|
||||||
|
- No health check after deployment
|
||||||
|
|
||||||
|
**After:**
|
||||||
|
- ✅ Depends on tests passing first (`needs: tests`)
|
||||||
|
- ✅ Uses reusable workflow pattern
|
||||||
|
- ✅ Adds health check after deployment
|
||||||
|
- ✅ Fails deployment if container doesn't become healthy
|
||||||
|
- ✅ Uses `actions/checkout@v4` consistently
|
||||||
|
- ✅ Adds cache-from for faster builds
|
||||||
|
|
||||||
|
### 3. Deploy Workflow - Dev (`deploy-dev.yml`)
|
||||||
|
**Before:**
|
||||||
|
- Deployed without waiting for tests
|
||||||
|
- Used manual `git clone`
|
||||||
|
- No clear success/failure indicators
|
||||||
|
|
||||||
|
**After:**
|
||||||
|
- ✅ Depends on tests passing first (`needs: tests`)
|
||||||
|
- ✅ Uses standardized `actions/checkout@v4`
|
||||||
|
- ✅ Better error messages with emojis (✅/❌)
|
||||||
|
- ✅ Runs smoke tests after deployment
|
||||||
|
- ✅ Fails if smoke tests don't pass
|
||||||
|
|
||||||
|
## Benefits
|
||||||
|
|
||||||
|
### Resource Efficiency
|
||||||
|
- Tests only run when code changes (not on README updates, etc.)
|
||||||
|
- Prevents multiple concurrent builds of the same code
|
||||||
|
- Uses build cache to speed up subsequent builds
|
||||||
|
|
||||||
|
### Safety
|
||||||
|
- **Zero broken deployments**: Tests must pass before deploy
|
||||||
|
- Health checks ensure container is actually working
|
||||||
|
- Smoke tests verify basic functionality
|
||||||
|
|
||||||
|
### Consistency
|
||||||
|
- All workflows use `actions/checkout@v4`
|
||||||
|
- Standardized error handling and logging
|
||||||
|
- Clear success/failure indicators
|
||||||
|
|
||||||
|
### Developer Experience
|
||||||
|
- Faster feedback on test failures
|
||||||
|
- Clear error messages when things fail
|
||||||
|
- No more "why did it deploy broken code?"
|
||||||
|
|
||||||
|
## Workflow Execution Flow
|
||||||
|
|
||||||
|
### Push to `dev` branch with dashboard changes:
|
||||||
|
```
|
||||||
|
1. Test workflow runs (backend + frontend tests)
|
||||||
|
2. If tests pass → Deploy to dev
|
||||||
|
3. If tests fail → No deployment (safe!)
|
||||||
|
4. After deploy → Health check + smoke tests
|
||||||
|
```
|
||||||
|
|
||||||
|
### Push to `main` branch with dashboard changes:
|
||||||
|
```
|
||||||
|
1. Test workflow runs (backend + frontend tests)
|
||||||
|
2. If tests pass → Deploy to production
|
||||||
|
3. If tests fail → No deployment (safe!)
|
||||||
|
4. After deploy → Health check
|
||||||
|
```
|
||||||
|
|
||||||
|
### Push to `dev` branch with README changes:
|
||||||
|
```
|
||||||
|
(No workflows run - saves resources!)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration Notes
|
||||||
|
|
||||||
|
- No breaking changes to existing workflows
|
||||||
|
- All existing functionality preserved
|
||||||
|
- Workflows are backward compatible
|
||||||
|
- Can be rolled back by reverting the commit
|
||||||
|
|
||||||
|
## Testing the Changes
|
||||||
|
|
||||||
|
After merging, test by:
|
||||||
|
1. Push a dashboard change to dev → should run tests then deploy
|
||||||
|
2. Push a README change to dev → should not trigger any workflows
|
||||||
|
3. Make tests fail → should block deployment
|
||||||
|
4. Check workflow logs for clear success/failure messages
|
||||||
@@ -11,14 +11,21 @@ concurrency:
|
|||||||
cancel-in-progress: true
|
cancel-in-progress: true
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
|
# Run tests first
|
||||||
|
tests:
|
||||||
|
name: Run Tests
|
||||||
|
uses: ./.gitea/workflows/test.yml
|
||||||
|
|
||||||
deploy-dev:
|
deploy-dev:
|
||||||
|
name: Deploy to Dev
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
|
needs: tests
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
run: |
|
uses: actions/checkout@v4
|
||||||
git clone --depth 1 http://gitea:3000/jimmy/nas-tools.git .
|
with:
|
||||||
git fetch --depth 1 origin ${{ github.sha }}
|
fetch-depth: 1
|
||||||
git checkout ${{ github.sha }}
|
|
||||||
- name: Warm mirror cache for base images
|
- name: Warm mirror cache for base images
|
||||||
run: |
|
run: |
|
||||||
set -u
|
set -u
|
||||||
@@ -55,10 +62,12 @@ jobs:
|
|||||||
curl -I https://pypi.tuna.tsinghua.edu.cn || echo "Tsinghua PyPI unreachable"
|
curl -I https://pypi.tuna.tsinghua.edu.cn || echo "Tsinghua PyPI unreachable"
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
- name: Sync runtime compose file
|
- name: Sync runtime compose file
|
||||||
run: |
|
run: |
|
||||||
mkdir -p /volume1/docker/nas-dashboard
|
mkdir -p /volume1/docker/nas-dashboard
|
||||||
cp dashboard/docker-compose.dev.yml /volume1/docker/nas-dashboard/docker-compose.dev.yml
|
cp dashboard/docker-compose.dev.yml /volume1/docker/nas-dashboard/docker-compose.dev.yml
|
||||||
|
|
||||||
- name: Deploy dev
|
- name: Deploy dev
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
@@ -80,13 +89,14 @@ jobs:
|
|||||||
# Manually connect to networks after container is created
|
# Manually connect to networks after container is created
|
||||||
docker network connect nas-dashboard_internal nas-dashboard-dev || true
|
docker network connect nas-dashboard_internal nas-dashboard-dev || true
|
||||||
docker network connect gitea_gitea nas-dashboard-dev || true
|
docker network connect gitea_gitea nas-dashboard-dev || true
|
||||||
|
|
||||||
- name: Wait for container to be healthy
|
- name: Wait for container to be healthy
|
||||||
run: |
|
run: |
|
||||||
echo "Waiting for container to be healthy..."
|
echo "Waiting for container to be healthy..."
|
||||||
for i in {1..30}; do
|
for i in {1..30}; do
|
||||||
STATUS=$(docker inspect --format='{{.State.Health.Status}}' nas-dashboard-dev 2>/dev/null || echo "starting")
|
STATUS=$(docker inspect --format='{{.State.Health.Status}}' nas-dashboard-dev 2>/dev/null || echo "starting")
|
||||||
if [ "$STATUS" = "healthy" ]; then
|
if [ "$STATUS" = "healthy" ]; then
|
||||||
echo "Container is healthy"
|
echo "✅ Container is healthy"
|
||||||
break
|
break
|
||||||
fi
|
fi
|
||||||
echo "Waiting... ($i/30) Status: $STATUS"
|
echo "Waiting... ($i/30) Status: $STATUS"
|
||||||
@@ -95,16 +105,18 @@ jobs:
|
|||||||
# Final check
|
# Final check
|
||||||
STATUS=$(docker inspect --format='{{.State.Health.Status}}' nas-dashboard-dev 2>/dev/null || echo "unknown")
|
STATUS=$(docker inspect --format='{{.State.Health.Status}}' nas-dashboard-dev 2>/dev/null || echo "unknown")
|
||||||
if [ "$STATUS" != "healthy" ]; then
|
if [ "$STATUS" != "healthy" ]; then
|
||||||
echo "Container failed to become healthy: $STATUS"
|
echo "❌ Container failed to become healthy: $STATUS"
|
||||||
docker logs nas-dashboard-dev --tail 50
|
docker logs nas-dashboard-dev --tail 50
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
- name: Run smoke tests
|
- name: Run smoke tests
|
||||||
run: |
|
run: |
|
||||||
cd /volume1/repos/nas-tools
|
cd /volume1/repos/nas-tools
|
||||||
bash scripts/smoke-test-dashboard.sh dev
|
bash scripts/smoke-test-dashboard.sh dev
|
||||||
if [ $? -ne 0 ]; then
|
if [ $? -ne 0 ]; then
|
||||||
echo "Smoke tests failed! Check logs above."
|
echo "❌ Smoke tests failed! Check logs above."
|
||||||
docker logs nas-dashboard-dev --tail 100
|
docker logs nas-dashboard-dev --tail 100
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
echo "✅ Smoke tests passed"
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
name: Deploy Dashboard
|
name: Deploy Dashboard (Main)
|
||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: [main]
|
branches: [main]
|
||||||
@@ -11,8 +11,15 @@ concurrency:
|
|||||||
cancel-in-progress: true
|
cancel-in-progress: true
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
|
# Run tests first
|
||||||
|
tests:
|
||||||
|
name: Run Tests
|
||||||
|
uses: ./.gitea/workflows/test.yml
|
||||||
|
|
||||||
deploy:
|
deploy:
|
||||||
|
name: Deploy to Production
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
|
needs: tests
|
||||||
container:
|
container:
|
||||||
volumes:
|
volumes:
|
||||||
- /var/packages/ContainerManager/target/usr/bin/docker:/usr/bin/docker
|
- /var/packages/ContainerManager/target/usr/bin/docker:/usr/bin/docker
|
||||||
@@ -22,10 +29,10 @@ jobs:
|
|||||||
uses: actions/checkout@v4
|
uses: actions/checkout@v4
|
||||||
with:
|
with:
|
||||||
fetch-depth: 1
|
fetch-depth: 1
|
||||||
|
|
||||||
- name: Warm mirror cache for base images
|
- name: Warm mirror cache for base images
|
||||||
run: |
|
run: |
|
||||||
set -u
|
set -u
|
||||||
# Use host.docker.internal or NAS IP to access registry mirror from container
|
|
||||||
mirror_host="100.78.131.124:5501"
|
mirror_host="100.78.131.124:5501"
|
||||||
|
|
||||||
if command -v curl >/dev/null 2>&1; then
|
if command -v curl >/dev/null 2>&1; then
|
||||||
@@ -40,6 +47,12 @@ jobs:
|
|||||||
|
|
||||||
prewarm_base_image() {
|
prewarm_base_image() {
|
||||||
image_ref="$1"
|
image_ref="$1"
|
||||||
|
# Skip if image already exists locally
|
||||||
|
if docker image inspect "${image_ref}" >/dev/null 2>&1; then
|
||||||
|
echo "Image ${image_ref} already exists locally, skipping pre-pull"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
mirror_ref="${mirror_host}/library/${image_ref}"
|
mirror_ref="${mirror_host}/library/${image_ref}"
|
||||||
echo "Attempting mirror pre-pull: ${mirror_ref}"
|
echo "Attempting mirror pre-pull: ${mirror_ref}"
|
||||||
if timeout 120 docker pull "${mirror_ref}"; then
|
if timeout 120 docker pull "${mirror_ref}"; then
|
||||||
@@ -57,13 +70,34 @@ jobs:
|
|||||||
run: |
|
run: |
|
||||||
set -e
|
set -e
|
||||||
export DOCKER_API_VERSION=1.43
|
export DOCKER_API_VERSION=1.43
|
||||||
if ! DOCKER_BUILDKIT=0 docker build -t nas-dashboard:latest dashboard/; then
|
if ! DOCKER_BUILDKIT=0 docker build --cache-from nas-dashboard:latest -t nas-dashboard:latest dashboard/; then
|
||||||
echo "Build failed. Checking for network issues..."
|
echo "Build failed. Checking for network issues..."
|
||||||
curl -I https://registry.npmmirror.com || echo "npmmirror unreachable"
|
curl -I https://registry.npmmirror.com || echo "npmmirror unreachable"
|
||||||
curl -I https://pypi.tuna.tsinghua.edu.cn || echo "Tsinghua PyPI unreachable"
|
curl -I https://pypi.tuna.tsinghua.edu.cn || echo "Tsinghua PyPI unreachable"
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
- name: Sync runtime compose file
|
- name: Sync runtime compose file
|
||||||
run: cp dashboard/docker-compose.yml /nas-dashboard/docker-compose.yml
|
run: cp dashboard/docker-compose.yml /nas-dashboard/docker-compose.yml
|
||||||
|
|
||||||
- name: Deploy
|
- name: Deploy
|
||||||
run: docker compose -f /nas-dashboard/docker-compose.yml up -d --pull never
|
run: |
|
||||||
|
set -e
|
||||||
|
export DOCKER_API_VERSION=1.43
|
||||||
|
docker compose -f /nas-dashboard/docker-compose.yml up -d --pull never
|
||||||
|
|
||||||
|
- name: Wait for container to be healthy
|
||||||
|
run: |
|
||||||
|
echo "Waiting for container to be healthy..."
|
||||||
|
for i in {1..30}; do
|
||||||
|
STATUS=$(docker inspect --format='{{.State.Health.Status}}' nas-dashboard 2>/dev/null || echo "starting")
|
||||||
|
if [ "$STATUS" = "healthy" ]; then
|
||||||
|
echo "✅ Container is healthy"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
echo "Waiting... ($i/30) Status: $STATUS"
|
||||||
|
sleep 2
|
||||||
|
done
|
||||||
|
echo "❌ Container failed to become healthy"
|
||||||
|
docker logs nas-dashboard --tail 50
|
||||||
|
exit 1
|
||||||
|
|||||||
+12
-13
@@ -3,8 +3,14 @@ name: Run Tests
|
|||||||
on:
|
on:
|
||||||
push:
|
push:
|
||||||
branches: [main, dev]
|
branches: [main, dev]
|
||||||
|
paths:
|
||||||
|
- 'dashboard/**'
|
||||||
|
- '.gitea/workflows/test.yml'
|
||||||
pull_request:
|
pull_request:
|
||||||
branches: [main]
|
branches: [main]
|
||||||
|
paths:
|
||||||
|
- 'dashboard/**'
|
||||||
|
- '.gitea/workflows/test.yml'
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
backend-tests:
|
backend-tests:
|
||||||
@@ -15,10 +21,9 @@ jobs:
|
|||||||
|
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
run: |
|
uses: actions/checkout@v4
|
||||||
git clone --depth 1 http://gitea:3000/jimmy/nas-tools.git .
|
with:
|
||||||
git fetch --depth 1 origin ${{ github.sha }}
|
fetch-depth: 1
|
||||||
git checkout ${{ github.sha }}
|
|
||||||
|
|
||||||
- name: Setup Python
|
- name: Setup Python
|
||||||
run: |
|
run: |
|
||||||
@@ -54,11 +59,9 @@ jobs:
|
|||||||
echo "Installing fresh dependencies..."
|
echo "Installing fresh dependencies..."
|
||||||
python3 -m venv venv
|
python3 -m venv venv
|
||||||
. venv/bin/activate
|
. venv/bin/activate
|
||||||
# Use Tsinghua PyPI mirror for faster downloads in China
|
|
||||||
pip install --cache-dir=$PIP_CACHE_DIR -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
|
pip install --cache-dir=$PIP_CACHE_DIR -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements.txt
|
||||||
pip install --cache-dir=$PIP_CACHE_DIR -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements-dev.txt
|
pip install --cache-dir=$PIP_CACHE_DIR -i https://pypi.tuna.tsinghua.edu.cn/simple -r requirements-dev.txt
|
||||||
pip install --cache-dir=$PIP_CACHE_DIR -i https://pypi.tuna.tsinghua.edu.cn/simple pytest-timeout pytest-cov
|
pip install --cache-dir=$PIP_CACHE_DIR -i https://pypi.tuna.tsinghua.edu.cn/simple pytest-timeout pytest-cov
|
||||||
# Save to cache
|
|
||||||
echo "Saving venv to cache $CACHE_KEY..."
|
echo "Saving venv to cache $CACHE_KEY..."
|
||||||
cp -a venv $CACHE_DIR/
|
cp -a venv $CACHE_DIR/
|
||||||
fi
|
fi
|
||||||
@@ -72,7 +75,6 @@ jobs:
|
|||||||
--junit-xml=test-results.xml \
|
--junit-xml=test-results.xml \
|
||||||
--cov-fail-under=49
|
--cov-fail-under=49
|
||||||
|
|
||||||
# Explicitly check exit code
|
|
||||||
if [ $? -ne 0 ]; then
|
if [ $? -ne 0 ]; then
|
||||||
echo "❌ Backend tests failed!"
|
echo "❌ Backend tests failed!"
|
||||||
exit 1
|
exit 1
|
||||||
@@ -103,10 +105,9 @@ jobs:
|
|||||||
|
|
||||||
steps:
|
steps:
|
||||||
- name: Checkout repository
|
- name: Checkout repository
|
||||||
run: |
|
uses: actions/checkout@v4
|
||||||
git clone --depth 1 http://gitea:3000/jimmy/nas-tools.git .
|
with:
|
||||||
git fetch --depth 1 origin ${{ github.sha }}
|
fetch-depth: 1
|
||||||
git checkout ${{ github.sha }}
|
|
||||||
|
|
||||||
- name: Setup Node.js
|
- name: Setup Node.js
|
||||||
run: |
|
run: |
|
||||||
@@ -140,7 +141,6 @@ jobs:
|
|||||||
else
|
else
|
||||||
echo "Installing fresh dependencies..."
|
echo "Installing fresh dependencies..."
|
||||||
npm ci
|
npm ci
|
||||||
# Save to cache
|
|
||||||
echo "Saving to cache $CACHE_KEY..."
|
echo "Saving to cache $CACHE_KEY..."
|
||||||
cp -a node_modules $CACHE_DIR/
|
cp -a node_modules $CACHE_DIR/
|
||||||
fi
|
fi
|
||||||
@@ -152,7 +152,6 @@ jobs:
|
|||||||
cd dashboard/frontend
|
cd dashboard/frontend
|
||||||
npm run test:coverage -- --reporter=verbose --run --pool=forks --poolOptions.forks.maxForks=2
|
npm run test:coverage -- --reporter=verbose --run --pool=forks --poolOptions.forks.maxForks=2
|
||||||
|
|
||||||
# Explicitly check exit code
|
|
||||||
if [ $? -ne 0 ]; then
|
if [ $? -ne 0 ]; then
|
||||||
echo "❌ Frontend tests failed!"
|
echo "❌ Frontend tests failed!"
|
||||||
exit 1
|
exit 1
|
||||||
|
|||||||
Reference in New Issue
Block a user