Sprint 002: Ingest duration & view count, FTS search, infinite scroll pagination, and transcript accordion
Deploy t-youtube / Build & Deploy (push) Failing after 1s

This commit is contained in:
Gan, Jimmy
2026-07-11 13:16:59 +08:00
parent 623dff149c
commit 022b024603
6 changed files with 207 additions and 91 deletions
+28 -26
View File
@@ -1,32 +1,34 @@
# Sprint 001Sync Reliability & Telegram Delivery
# Sprint 002UX Overhaul, Full-Text Search & Metadata Enrichment
**Dates:** Active
**Goal:** Overhaul the t-youtube reading experience by completing the search workflow, adding transcript visibility, fetching duration/views metadata, and implementing cursor-based/offset pagination (infinite scroll).
## Goals
---
- Fix daily sync on NAS
- Verify Telegram delivery pipeline
- Stabilize subscription fetching behind GFW
## T2.1 — Full-Text Search Frontend Integration
- **Files:** `frontend/src/routes/YoutubeDashboard.svelte`
- **Done means:** Search queries directly populate the main video list in the sidebar (rather than a temporary dropdown), and clearing the search resets the view.
- **Verify by:**
1. Type search query -> video list dynamically updates with FTS results from backend.
2. Click a search result -> detail view loads it.
3. Clear query/blur search -> list reverts to standard tab view.
## Tasks
## T2.2 — Transcript Accordion in Details View
- **Files:** `frontend/src/routes/YoutubeDashboard.svelte`
- **Done means:** Add an expandable accordion below the "AI Digest" section to view the raw video transcript. If transcript is long, render in scrollable container.
- **Verify by:**
1. Click video with transcript -> "Show Full Transcript" accordion appears.
2. Click to expand -> raw transcript displays correctly.
### T1: Fix daily sync cron
## T2.3 — Ingest Duration and View Count Metadata
- **Files:** `backend/fetcher.py`, `backend/tests/test_fetcher.py`
- **Done means:** Update `fetcher.py` videos.list API call to request `part="snippet,contentDetails,statistics"`. Parse ISO 8601 duration into integer seconds, and store duration + view count in SQLite db. Update unit tests to include mock response fields.
- **Verify by:**
1. Run `./run_tests.sh` to ensure fetcher tests pass.
2. Sync subscriptions -> inspect database to confirm `duration` and `view_count` are populated for new videos.
**Description:**
`t-youtube-daily-sync` errored last run. Determine if `run_sync.py` is the active pipeline or if VPS-based `fetch_and_sync.sh` replaced it. Fix whichever is active.
**ACs:**
- [x] Daily sync completes without errors (verified: last run 2026-07-11 03:16 OK)
- [x] New videos appear in dashboard (verified: DB populated, latest video ingested 2026-07-10)
**Impl Notes:**
Check NAS cron logs. VPS sync fires every 6h.
### T2: Verify tg-group-filter delivery
**Description:**
Cron switched from `deliver: local` to `deliver: all`. Confirm Telegram delivery works for filtered results.
**ACs:**
- [x] Telegram receives filtered video digest (tg-group-filter running every 30m, verified delivering)
- [x] No duplicate deliveries (single digest per cycle)
## T2.4 — Cursor/Offset Pagination (Infinite Scroll)
- **Files:** `backend/main.py`, `frontend/src/routes/YoutubeDashboard.svelte`
- **Done means:** Add pagination queries to `/api/videos` backend endpoint (e.g. `offset` and `limit`). In Svelte frontend, implement scroll listener on the sidebar video list to trigger next page load when reaching bottom.
- **Verify by:**
1. Load pending tab -> initially loads 50 videos.
2. Scroll to bottom of sidebar -> next 50 videos append automatically.