t-youtube/docs/progress.md

t-youtube Progress Tracking

Session 2026-07-09: Agent Queue, Notes Migration, Architecture Documentation

Completed

• Agent Queue Feature:
  • Added agent_queue table to SQLite DB
  • Implemented POST/GET/DELETE endpoints in main.py
  • Created agent_worker.py skeleton for processing pending items
  • Frontend "Agent Queue" tab (Svelte 5) with status indicators
• Notes Migration:
  • Moved notes from localStorage to server-side DB (video_notes table)
  • Added POST/GET/DELETE endpoints for notes
  • Implemented non-blocking localStorage migration script
• Database Optimization:
  • Added indexes on video_id for agent_queue and video_notes tables
• Authelia Configuration:
  • Attempted passwordless mode (failed due to unsupported key in Authelia v4)
  • Reverted to standard 2FA (WebAuthn/TOTP)
• Documentation:
  • Created docs/ folder with architecture.md and progress.md
  • Documented all API endpoints, DB schema, infrastructure, and known issues

In Progress

• Agent Worker Logic:
  • agent_worker.py needs actual LLM/feature-creation logic
  • Pending: Define workflow for processing queued items
• YouTube Sync Monitoring:
  • Daily cron at 3 AM is active
  • Pending: Monitor next run for errors/progress

Planned

• Channel Categorization:
  • ~119 channels remain uncategorized
  • Consider moving AUTO_CATEGORIES to server-side JSON config
• Transcript Fallback:
  • Implement browser-based scraping (playwright/selenium) for the remaining 10%
  • High complexity/resource usage trade-off
• Performance Monitoring:
  • Add health check endpoints for sync status
  • Implement error logging/reporting

Session 2026-07-05 to 2026-07-08: Initial Development

Key Milestones

• YouTube Data API v3 + RSS Hybrid: Replaced yt-dlp/cookies approach
  • Free, unlimited sync (~50 units/sync for 200+ channels)
  • 2414 new videos discovered in first run
• AI Summarization: DeepSeek API integration (was Claude/bytecatcode)
  • Chinese content → Chinese summary; English content → English summary
  • ~2.1s per video, ~$0.001 each
• Frontend Overhaul:
  • AI Digest as hero element
  • Keyboard navigation (j/k/↑↓, r/m, s, n, Enter, ?)
  • Dark/light mode toggle
  • Channel filter dropdown with auto-generated categories
  • Auto-read after 3s silent mark
• Agent Queue Integration:
  • "Save to Agent" feature: notes ARE prompts for Hermes agents
  • Queue tab with pending/processing/done status
  • Cross-device note sync (localStorage → server DB)

Technical Decisions

1. API Token Auth: Bearer token for API writes, Authelia SSO for frontend
2. SQLite: Simple, reliable, no external dependencies for single-user setup
3. DeepSeek API: Cost-effective for summarization (~$0.001/video)
4. Svelte 5: Reactivity model, keyboard workflow, small bundle size
5. RSS + YouTube Data API: Free tier sufficient, avoids cookie/yt-dlp maintenance

Session 2026-07-04: NAS & Authelia Setup

Completed

• NAS Caddy port 443 conflict resolved (DSM nginx)
• dnsmasq → VPS Caddy Tailscale IP (100.109.198.126)
• Authelia WebAuthn enabled, passkey registered
• TOTP reset for jimmy, default 2FA set to WebAuthn
• Caddy SNI bug workaround (global TLS config, port 8443 fallback)

Known Issues & Resolutions

Authelia Passwordless Failure

• Issue: webauthn.passwordless: true caused boot loop (configuration key not expected)
• Root Cause: Unsupported configuration key in current Authelia version
• Resolution: Reverted to standard 2FA (WebAuthn/TOTP)

YouTube Transcript Blocking

• Issue: 90% of videos have transcripts; 10% fail due to Oracle Cloud IP blocking
• Root Cause: YouTube IP-based restrictions
• Resolution: Fallback to video description summaries; consider browser-based scraping later

NAS Caddy SNI Bug

• Issue: NAS Caddy serves first cached cert for all SNI names
• Root Cause: Per-site tls { ... } blocks conflict with global auto_https policy
• Resolution: Global TLS config + dnsmasq → VPS Caddy Tailscale IP

Metrics & Performance

• Videos in DB: ~2414+ (initial sync)
• Channels: ~229 subscribed
• Sync Time: ~2-3 minutes for all channels
• Summarization: ~2.1s/video, ~$0.001 each via DeepSeek
• Frontend Load Time: 0.14-0.22s (API responses)
• Daily API Usage: ~50 units (well within 10,000/day free tier)