t-youtube/docs/troubleshooting.md

# t-youtube Troubleshooting Guide

## Common Issues

### 1. Site Down / 502 Bad Gateway
- **Cause**: t-youtube container not running or Authelia down
- **Check**:
  ```bash
  ssh nas 'docker ps | grep t-youtube'
  ssh nas 'docker ps | grep authelia'
  ```
- **Fix**:
  ```bash
  cd /volume1/docker/t-youtube && /usr/local/bin/docker compose up -d
  cd /volume1/docker/authelia && /usr/local/bin/docker compose restart authelia
  ```

### 2. Authelia Loop / Login Fails
- **Cause**: WebAuthn/TOTP misconfiguration or session expired
- **Check Authelia logs**:
  ```bash
  ssh nas 'docker logs authelia --tail 20'
  ```
- **Fix**:
  - Clear browser cache/cookies
  - Reset WebAuthn via Authelia DB (see below)
  - Check dnsmasq config (should point to VPS Tailscale IP)

### 3. YouTube Data API Quota Exceeded
- **Symptom**: Fetcher returns empty results or 403 errors
- **Cause**: Daily quota limit reached (10,000 units)
- **Check**:
  ```bash
  ssh nas 'docker logs t-youtube 2>&1 | grep -i "quota\|403"'
  ```
- **Fix**:
  - Wait until next day (quota resets at 00:00 UTC)
  - Reduce number of channels or increase sync interval
  - Monitor usage in Google Cloud Console

### 4. Summarization Failing
- **Symptom**: Videos show "No summary" or generic descriptions
- **Cause**: DeepSeek API key invalid or rate limited
- **Check**:
  ```bash
  ssh nas 'docker logs t-youtube 2>&1 | grep -i "summariz\|deepseek"'
  ```
- **Fix**:
  - Verify `DEEPSEEK_API_KEY` in `.env`
  - Test API key manually:
    ```bash
    curl -s -X POST https://api.deepseek.com/chat/completions \
      -H "Authorization: Bearer ***
      -H "Content-Type: application/json" \
      -d '{"model": "deepseek-chat", "messages": [{"role": "user", "content": "test"}]}'
    ```

### 5. Database Locked/Corrupted
- **Symptom**: API returns 500 errors or hangs
- **Cause**: Concurrent write operations or unexpected shutdown
- **Fix**:
  ```bash
  cd /volume1/docker/t-youtube/backend
  cp t_youtube.db t_youtube.db.backup
  sqlite3 t_youtube.db "PRAGMA integrity_check;"
  # If corrupted, restore from backup or recreate
  ```

### 6. Caddy Config Issues
- **Symptom**: 502/503 errors, wrong cert, or domain unreachable
- **Check**:
  ```bash
  ssh caddy-vps "sudo caddy validate --config /etc/caddy/Caddyfile"
  ssh nas 'docker exec caddy caddy validate --config /etc/caddy/Caddyfile'
  ```
- **Fix**:
  - Validate config
  - Reload/restart Caddy
  - Check TLS certs:
    ```bash
    echo | openssl s_client -connect 161.33.182.109:443 -servername yt.jimmygan.com 2>&1 | openssl x509 -noout -subject -dates
    ```

## API Debugging

### Check Video List
```bash
curl -s https://yt.jimmygan.com/api/videos | python3 -m json.tool | head -20
```

### Check Agent Queue
```bash
curl -s -H "Authorization: Bearer *** https://yt.jimmygan.com/api/agent-queue
```

### Check Notes
```bash
curl -s -H "Authorization: Bearer *** https://yt.jimmygan.com/api/notes
```

### Check Channel List
```bash
curl -s -H "Authorization: Bearer *** https://yt.jimmygan.com/api/channels
```

### Test Summarization
```bash
curl -s -X POST -H "Authorization: Bearer ***
  -H "Content-Type: application/json" \
  -d '{"video_id": "<video_id>"}' \
  https://yt.jimmygan.com/api/agent-queue/<video_id>/summarize
```

## Database Queries

### Count Videos by Status
```sql
SELECT status, COUNT(*) FROM videos GROUP BY status;
```

### Count Agent Queue Items by Status
```sql
SELECT status, COUNT(*) FROM agent_queue GROUP BY status;
```

### Find Videos Without Summaries
```sql
SELECT video_id, title FROM videos WHERE summary IS NULL OR summary = '';
```

### Check DB Size
```bash
ls -lh /volume1/docker/t-youtube/backend/t_youtube.db
```

## Logs

### t-youtube Logs
```bash
ssh nas 'docker logs t-youtube --tail 50'
ssh nas 'docker logs t-youtube --tail 50 --since 1h'
```

### Authelia Logs
```bash
ssh nas 'docker logs authelia --tail 50'
```

### Sync Logs
```bash
ssh nas 'cat /var/log/t-youtube-sync.log'
ssh nas 'tail -100 /var/log/t-youtube-sync.log'
```

### Caddy Logs (VPS)
```bash
ssh caddy-vps "sudo journalctl -u caddy --since '1 hour ago'"
ssh caddy-vps "sudo journalctl -u caddy --no-pager -n 30"
```

## Recovery Procedures

### Full System Restart
```bash
# 1. Restart services
cd /volume1/docker/authelia && /usr/local/bin/docker compose restart authelia
cd /volume1/docker/t-youtube && /usr/local/bin/docker compose up -d

# 2. Verify
curl -s https://yt.jimmygan.com/api/health
curl -s https://auth.jimmygan.com/api/health
```

### Database Backup & Restore
```bash
# Backup
cp /volume1/docker/t-youtube/backend/t_youtube.db /tmp/t-youtube-backup-$(date +%Y%m%d).db

# Restore
cp /tmp/t-youtube-backup-20260709.db /volume1/docker/t-youtube/backend/t_youtube.db
cd /volume1/docker/t-youtube && /usr/local/bin/docker compose restart t-youtube
```

### Authelia User Reset (if locked out)
```bash
# Stop Authelia
ssh nas 'docker stop authelia'

# Reset TOTP for jimmy
ssh nas 'sqlite3 /volume1/docker/authelia/config/db.sqlite3 "DELETE FROM totp_configurations WHERE username='\'jimmy'\'';"'

# Restart Authelia
ssh nas 'docker start authelia'
```