feat: S07 operations hardening and OpenConnector integration
Deploy Claude Dev (Dev) / deploy-claude-dev-dev (push) Failing after 2m3s
Deploy Dashboard (Main) / Backend Tests (push) Successful in 3m7s
Deploy Dashboard (Main) / Frontend Tests (push) Successful in 6s
Deploy Dashboard (Main) / Deploy to Production (push) Has been cancelled
Deploy Dashboard (Main) / Build Main Image (push) Has been cancelled
Deploy Dashboard (Dev) / Backend Tests (push) Successful in 1m18s
Deploy Dashboard (Dev) / Frontend Tests (push) Successful in 51s
Deploy Dashboard (Dev) / Build Dev Image (push) Successful in 1m42s
Deploy Dashboard (Dev) / Deploy to Dev (push) Successful in 58s
Deploy Claude Dev (Dev) / deploy-claude-dev-dev (push) Failing after 2m3s
Deploy Dashboard (Main) / Backend Tests (push) Successful in 3m7s
Deploy Dashboard (Main) / Frontend Tests (push) Successful in 6s
Deploy Dashboard (Main) / Deploy to Production (push) Has been cancelled
Deploy Dashboard (Main) / Build Main Image (push) Has been cancelled
Deploy Dashboard (Dev) / Backend Tests (push) Successful in 1m18s
Deploy Dashboard (Dev) / Frontend Tests (push) Successful in 51s
Deploy Dashboard (Dev) / Build Dev Image (push) Successful in 1m42s
Deploy Dashboard (Dev) / Deploy to Dev (push) Successful in 58s
This commit is contained in:
@@ -47,6 +47,9 @@ log "=== Backup started ==="
|
||||
run_backup "Gitea DB" docker exec gitea-db pg_dump -U gitea -f /tmp/gitea_dump.sql gitea
|
||||
docker cp gitea-db:/tmp/gitea_dump.sql "$BACKUP_DIR/gitea_db_$DATE.sql" 2>/dev/null
|
||||
|
||||
run_backup "OPC DB" docker exec gitea-db pg_dump -U gitea -f /tmp/opc_dump.sql opc
|
||||
docker cp gitea-db:/tmp/opc_dump.sql "$BACKUP_DIR/opc_db_$DATE.sql" 2>/dev/null
|
||||
|
||||
run_backup "Immich DB" docker exec immich-db pg_dump -U postgres -f /tmp/immich_dump.sql immich
|
||||
docker cp immich-db:/tmp/immich_dump.sql "$BACKUP_DIR/immich_db_$DATE.sql" 2>/dev/null
|
||||
|
||||
@@ -55,10 +58,11 @@ CONFIGS=()
|
||||
for p in \
|
||||
/volume1/docker/nas-dashboard/.env \
|
||||
/volume1/docker/nas-dashboard/auth.json \
|
||||
/volume1/docker/nas-dashboard/rbac.json \
|
||||
/volume1/docker/nas-dashboard/audit.log \
|
||||
/volume1/docker/gitea/conf \
|
||||
/volume1/docker/immich/.env \
|
||||
/volume1/docker/navidrome/data/navidrome.db \
|
||||
/volume1/docker/openclaw/data \
|
||||
/volume1/docker/n8n/docker-compose.yml; do
|
||||
[ -e "$p" ] && CONFIGS+=("$p")
|
||||
done
|
||||
|
||||
@@ -3,6 +3,8 @@ services:
|
||||
image: claude-dev:${CLAUDE_DEV_IMAGE_TAG:?CLAUDE_DEV_IMAGE_TAG is required}
|
||||
container_name: claude-dev
|
||||
restart: unless-stopped
|
||||
mem_limit: 2g
|
||||
cpus: 2.0
|
||||
network_mode: host
|
||||
stdin_open: true
|
||||
tty: true
|
||||
|
||||
@@ -72,6 +72,7 @@ EXTERNAL_SERVICES = {
|
||||
"vaultwarden": os.environ.get("VAULTWARDEN_URL", "https://vault.jimmygan.com"),
|
||||
"n8n": os.environ.get("N8N_URL", "https://n8n.jimmygan.com"),
|
||||
"speedtest": os.environ.get("SPEEDTEST_URL", "https://speed.jimmygan.com"),
|
||||
"openconnector": os.environ.get("OPENCONNECTOR_URL", "https://connector.jimmygan.com"),
|
||||
}
|
||||
|
||||
# Network addresses (used by frontend for LAN/Tailscale direct access)
|
||||
|
||||
@@ -143,12 +143,13 @@ async def security_headers(request: Request, call_next):
|
||||
|
||||
# Audit logger
|
||||
import os
|
||||
from logging.handlers import RotatingFileHandler
|
||||
|
||||
_audit_log_path = os.path.join(VOLUME_ROOT, "docker/nas-dashboard/audit.log")
|
||||
os.makedirs(os.path.dirname(_audit_log_path), exist_ok=True)
|
||||
_audit_logger = logging.getLogger("audit")
|
||||
_audit_logger.setLevel(logging.INFO)
|
||||
_audit_handler = logging.FileHandler(_audit_log_path)
|
||||
_audit_handler = RotatingFileHandler(_audit_log_path, maxBytes=10 * 1024 * 1024, backupCount=5)
|
||||
_audit_handler.setFormatter(logging.Formatter("%(message)s"))
|
||||
_audit_logger.addHandler(_audit_handler)
|
||||
_audit_logger.propagate = False
|
||||
@@ -229,7 +230,48 @@ async def monitor_containers():
|
||||
|
||||
@app.get("/api/health")
|
||||
async def health():
|
||||
return {"status": "ok"}
|
||||
import shutil
|
||||
import docker
|
||||
import logging
|
||||
from fastapi.responses import JSONResponse
|
||||
from fastapi import status
|
||||
from db import opc_db
|
||||
from config import DOCKER_HOST, VOLUME_ROOT
|
||||
|
||||
db_ok = False
|
||||
try:
|
||||
pool = await opc_db.get_pool()
|
||||
async with pool.acquire() as conn:
|
||||
await conn.execute("SELECT 1")
|
||||
db_ok = True
|
||||
except Exception as e:
|
||||
logging.getLogger(__name__).error("Health check DB connection failed: %s", e)
|
||||
|
||||
docker_ok = False
|
||||
try:
|
||||
client = docker.DockerClient(base_url=DOCKER_HOST)
|
||||
docker_ok = client.ping()
|
||||
except Exception as e:
|
||||
logging.getLogger(__name__).error("Health check Docker connection failed: %s", e)
|
||||
|
||||
free_gb = 0.0
|
||||
try:
|
||||
total, used, free = shutil.disk_usage(VOLUME_ROOT)
|
||||
free_gb = round(free / (1024 ** 3), 2)
|
||||
except Exception as e:
|
||||
logging.getLogger(__name__).error("Health check disk usage failed: %s", e)
|
||||
|
||||
status_code = status.HTTP_200_OK if (db_ok and docker_ok) else status.HTTP_503_SERVICE_UNAVAILABLE
|
||||
|
||||
return JSONResponse(
|
||||
status_code=status_code,
|
||||
content={
|
||||
"status": "ok" if (db_ok and docker_ok) else "degraded",
|
||||
"db": "connected" if db_ok else "disconnected",
|
||||
"docker": "healthy" if docker_ok else "unhealthy",
|
||||
"disk": {"free_gb": free_gb}
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@app.get("/api/client-ip")
|
||||
|
||||
@@ -3,6 +3,8 @@ services:
|
||||
image: tecnativa/docker-socket-proxy
|
||||
container_name: docker-socket-proxy
|
||||
restart: unless-stopped
|
||||
mem_limit: 128m
|
||||
cpus: 0.2
|
||||
labels:
|
||||
- "com.centurylinklabs.watchtower.enable=true"
|
||||
environment:
|
||||
@@ -22,10 +24,11 @@ services:
|
||||
max-file: "3"
|
||||
|
||||
dashboard:
|
||||
# build: .
|
||||
image: nas-dashboard:latest
|
||||
container_name: nas-dashboard
|
||||
restart: unless-stopped
|
||||
mem_limit: 2g
|
||||
cpus: 1.5
|
||||
ports:
|
||||
- "127.0.0.1:4000:4000"
|
||||
environment:
|
||||
|
||||
@@ -56,6 +56,7 @@
|
||||
{ label: "Vaultwarden", section: "tools", remoteHref: "https://vault.jimmygan.com", description: "Password manager" },
|
||||
{ label: "n8n", section: "tools", remoteHref: "https://n8n.jimmygan.com", description: "Workflow automation" },
|
||||
{ label: "Speedtest", section: "tools", remoteHref: "https://speed.jimmygan.com", description: "Network speed test" },
|
||||
{ label: "OpenConnector", section: "tools", remoteHref: "https://connector.jimmygan.com", description: "Credential broker for AI agents" },
|
||||
// Settings
|
||||
{ id: "settings", label: "Settings", section: "tools", description: "Dashboard configuration" },
|
||||
];
|
||||
|
||||
@@ -63,6 +63,7 @@
|
||||
{ label: "Vaultwarden", remoteHref: "https://vault.jimmygan.com", icon: "lock", external: true },
|
||||
{ label: "n8n", remoteHref: "https://n8n.jimmygan.com", icon: "n8n", external: true },
|
||||
{ label: "Speedtest", remoteHref: "https://speed.jimmygan.com", icon: "speedtest", external: true },
|
||||
{ label: "OpenConnector", remoteHref: "https://connector.jimmygan.com", port: 3002, icon: "link", external: true },
|
||||
];
|
||||
|
||||
const defaultTools2 = [];
|
||||
@@ -275,6 +276,8 @@
|
||||
<svg class="w-[16px] h-[16px]" fill="none" viewBox="0 0 24 24" stroke="currentColor" stroke-width="1.8"><path stroke-linecap="round" stroke-linejoin="round" d="M11.42 15.17l-7.05 7.05a2 2 0 01-2.83 0l-.09-.09a2 2 0 010-2.83l7.05-7.05m4.24-4.24l1.41-1.41a4 4 0 015.66 0l.09.09a4 4 0 010 5.66l-1.41 1.41M14 14l-3-3m2 8l5-5" /></svg>
|
||||
{:else if icon === "robot"}
|
||||
<svg class="w-[16px] h-[16px]" fill="none" viewBox="0 0 24 24" stroke="currentColor" stroke-width="1.8"><path stroke-linecap="round" stroke-linejoin="round" d="M12 2v4M8 12h8M8 16h4m-2-8h0m0 0c-4 0-7 2-7 6s3 6 7 6 7-2 7-6-3-6-7-6zM6 8V6m12 2V6" /></svg>
|
||||
{:else if icon === "link"}
|
||||
<svg class="w-[16px] h-[16px]" fill="none" viewBox="0 0 24 24" stroke="currentColor" stroke-width="1.8"><path stroke-linecap="round" stroke-linejoin="round" d="M13.828 10.172a4 4 0 00-5.656 0l-4 4a4 4 0 105.656 5.656l1.102-1.101m-.758-4.899a4 4 0 005.656 0l4-4a4 4 0 00-5.656-5.656l-1.1 1.1" /></svg>
|
||||
{/if}
|
||||
</span>
|
||||
{/snippet}
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
# Sprint 07 — Operations Hardening
|
||||
|
||||
**Status:** Completed ✅
|
||||
**Goal:** Fix critical gaps in system backups, log rotation, container resource management, and health monitoring.
|
||||
|
||||
---
|
||||
|
||||
## S07.1 — Implement audit log rotation [Completed ✅]
|
||||
|
||||
- **Files:** `dashboard/backend/main.py`
|
||||
- **Action:** Replaced `logging.FileHandler` with `logging.handlers.RotatingFileHandler` (10MB limit, 5 backup files).
|
||||
- **Verification:** Standard requests function properly; Python syntax checked and tests passed.
|
||||
|
||||
## S07.2 — Fix database backups (Add OPC & remove dead paths) [Completed ✅]
|
||||
|
||||
- **Files:** `backup.sh`
|
||||
- **Action:**
|
||||
1. Added `OPC DB` dump using pg_dump.
|
||||
2. Added `/volume1/docker/nas-dashboard/rbac.json` and `audit.log` config paths to the configs backup block.
|
||||
3. Removed the deprecated `/volume1/docker/openclaw/data` path.
|
||||
- **Verification:** Shell script verified for syntax.
|
||||
|
||||
## S07.3 — Enhance health check endpoint [Completed ✅]
|
||||
|
||||
- **Files:** `dashboard/backend/main.py`
|
||||
- **Action:** Enhanced `/api/health` to dynamically query database pool availability, docker ping status, and disk space usage. Returns HTTP 503 if any critical service is down.
|
||||
- **Verification:** Integration test suite verified that endpoint changes compile and execute cleanly.
|
||||
|
||||
## S07.4 — Set Docker container resource limits [Completed ✅]
|
||||
|
||||
- **Files:** `dashboard/docker-compose.yml`, `claude-dev/docker-compose.yml`
|
||||
- **Action:** Added container memory limits and CPU limits (`mem_limit` and `cpus` service-level directives).
|
||||
- **Verification:** Compose YAML structures verified via linter and schema checker.
|
||||
Reference in New Issue
Block a user