Gan, Jimmy
103 lines
4.4 KiB
Markdown
103 lines
4.4 KiB
Markdown
# t-youtube Progress Tracking
|
|
|
|
## Session 2026-07-09: Agent Queue, Notes Migration, Architecture Documentation
|
|
|
|
### Completed
|
|
- [x] **Agent Queue Feature**:
|
|
- Added `agent_queue` table to SQLite DB
|
|
- Implemented `POST/GET/DELETE` endpoints in `main.py`
|
|
- Created `agent_worker.py` skeleton for processing pending items
|
|
- Frontend "Agent Queue" tab (Svelte 5) with status indicators
|
|
- [x] **Notes Migration**:
|
|
- Moved notes from localStorage to server-side DB (`video_notes` table)
|
|
- Added `POST/GET/DELETE` endpoints for notes
|
|
- Implemented non-blocking localStorage migration script
|
|
- [x] **Database Optimization**:
|
|
- Added indexes on `video_id` for `agent_queue` and `video_notes` tables
|
|
- [x] **Authelia Configuration**:
|
|
- Attempted passwordless mode (failed due to unsupported key in Authelia v4)
|
|
- Reverted to standard 2FA (WebAuthn/TOTP)
|
|
- [x] **Documentation**:
|
|
- Created `docs/` folder with `architecture.md` and `progress.md`
|
|
- Documented all API endpoints, DB schema, infrastructure, and known issues
|
|
|
|
### In Progress
|
|
- [ ] **Agent Worker Logic**:
|
|
- `agent_worker.py` needs actual LLM/feature-creation logic
|
|
- Pending: Define workflow for processing queued items
|
|
- [ ] **YouTube Sync Monitoring**:
|
|
- Daily cron at 3 AM is active
|
|
- Pending: Monitor next run for errors/progress
|
|
|
|
### Planned
|
|
- [ ] **Channel Categorization**:
|
|
- ~119 channels remain uncategorized
|
|
- Consider moving `AUTO_CATEGORIES` to server-side JSON config
|
|
- [ ] **Transcript Fallback**:
|
|
- Implement browser-based scraping (playwright/selenium) for the remaining 10%
|
|
- High complexity/resource usage trade-off
|
|
- [ ] **Performance Monitoring**:
|
|
- Add health check endpoints for sync status
|
|
- Implement error logging/reporting
|
|
|
|
## Session 2026-07-05 to 2026-07-08: Initial Development
|
|
|
|
### Key Milestones
|
|
- [x] **YouTube Data API v3 + RSS Hybrid**: Replaced yt-dlp/cookies approach
|
|
- Free, unlimited sync (~50 units/sync for 200+ channels)
|
|
- 2414 new videos discovered in first run
|
|
- [x] **AI Summarization**: DeepSeek API integration (was Claude/bytecatcode)
|
|
- Chinese content → Chinese summary; English content → English summary
|
|
- ~2.1s per video, ~$0.001 each
|
|
- [x] **Frontend Overhaul**:
|
|
- AI Digest as hero element
|
|
- Keyboard navigation (`j/k/↑↓`, `r/m`, `s`, `n`, `Enter`, `?`)
|
|
- Dark/light mode toggle
|
|
- Channel filter dropdown with auto-generated categories
|
|
- Auto-read after 3s silent mark
|
|
- [x] **Agent Queue Integration**:
|
|
- "Save to Agent" feature: notes ARE prompts for Hermes agents
|
|
- Queue tab with pending/processing/done status
|
|
- Cross-device note sync (localStorage → server DB)
|
|
|
|
### Technical Decisions
|
|
1. **API Token Auth**: Bearer token for API writes, Authelia SSO for frontend
|
|
2. **SQLite**: Simple, reliable, no external dependencies for single-user setup
|
|
3. **DeepSeek API**: Cost-effective for summarization (~$0.001/video)
|
|
4. **Svelte 5**: Reactivity model, keyboard workflow, small bundle size
|
|
5. **RSS + YouTube Data API**: Free tier sufficient, avoids cookie/yt-dlp maintenance
|
|
|
|
## Session 2026-07-04: NAS & Authelia Setup
|
|
|
|
### Completed
|
|
- [x] NAS Caddy port 443 conflict resolved (DSM nginx)
|
|
- [x] dnsmasq → VPS Caddy Tailscale IP (100.109.198.126)
|
|
- [x] Authelia WebAuthn enabled, passkey registered
|
|
- [x] TOTP reset for jimmy, default 2FA set to WebAuthn
|
|
- [x] Caddy SNI bug workaround (global TLS config, port 8443 fallback)
|
|
|
|
## Known Issues & Resolutions
|
|
|
|
### Authelia Passwordless Failure
|
|
- **Issue**: `webauthn.passwordless: true` caused boot loop (`configuration key not expected`)
|
|
- **Root Cause**: Unsupported configuration key in current Authelia version
|
|
- **Resolution**: Reverted to standard 2FA (WebAuthn/TOTP)
|
|
|
|
### YouTube Transcript Blocking
|
|
- **Issue**: 90% of videos have transcripts; 10% fail due to Oracle Cloud IP blocking
|
|
- **Root Cause**: YouTube IP-based restrictions
|
|
- **Resolution**: Fallback to video description summaries; consider browser-based scraping later
|
|
|
|
### NAS Caddy SNI Bug
|
|
- **Issue**: NAS Caddy serves first cached cert for all SNI names
|
|
- **Root Cause**: Per-site `tls { ... }` blocks conflict with global auto_https policy
|
|
- **Resolution**: Global TLS config + dnsmasq → VPS Caddy Tailscale IP
|
|
|
|
## Metrics & Performance
|
|
- **Videos in DB**: ~2414+ (initial sync)
|
|
- **Channels**: ~229 subscribed
|
|
- **Sync Time**: ~2-3 minutes for all channels
|
|
- **Summarization**: ~2.1s/video, ~$0.001 each via DeepSeek
|
|
- **Frontend Load Time**: 0.14-0.22s (API responses)
|
|
- **Daily API Usage**: ~50 units (well within 10,000/day free tier)
|