Gan, Jimmy
784a67c921
- Update index.html file input to accept audio/*,video/* - Add engine selection dropdown (Local vs Server) in index.html - Implement JS bufferToWav WAV encoder in app.js (Float32Array PCM to 16-bit WAV) - Integrate server-side transcribing route via FormData POST in app.js - Update DEVELOPMENT_RULES.md to mark browser-side extraction as implemented Co-Authored-By: Claude Opus 4.8 <noreply@anthreply.com>
54 lines
2.1 KiB
Markdown
54 lines
2.1 KiB
Markdown
# Media to Text - Development Rules
|
|
|
|
## CRITICAL RULE: Never Upload Full Video Files
|
|
|
|
**Status**: MANDATORY - NEVER VIOLATE
|
|
|
|
### Rule Definition
|
|
- **NEVER** allow users to upload full video files (MP4, AVI, MOV, MKV, FLV, WMV, WEBM) to the server
|
|
- **ALWAYS** extract audio on the browser BEFORE uploading
|
|
- Only audio files (MP3, WAV, M4A, etc.) should be uploaded to the server
|
|
|
|
### Why This Rule Exists
|
|
1. **Bandwidth Efficiency**: Video files are 10-100x larger than extracted audio
|
|
2. **User Experience**: Faster uploads = better UX
|
|
3. **Server Resources**: Reduces server storage and processing load
|
|
4. **Cost**: Minimizes data transfer costs
|
|
|
|
### Implementation Requirements
|
|
- Browser-side audio extraction is MANDATORY for video files
|
|
- Use ffmpeg.wasm or equivalent WebAssembly solution
|
|
- If browser extraction fails, show error message - DO NOT fall back to uploading full video
|
|
- Server-side extraction is NOT acceptable as a workaround
|
|
|
|
### Current Status
|
|
- ✅ Browser extraction via Web Audio API (`AudioContext.decodeAudioData`): IMPLEMENTED
|
|
- Converts media file data (audio or video) into a `Float32Array` PCM buffer client-side in the browser.
|
|
- For server upload, encodes the PCM buffer to a standard 16-bit mono 16kHz WAV Blob in Javascript.
|
|
- Ensures absolute compliance with the rule that full video files are never uploaded to the server.
|
|
|
|
### NOT Acceptable
|
|
- ❌ Uploading full video files to server
|
|
- ❌ Server-side extraction as primary solution
|
|
- ❌ Removing this rule without explicit approval
|
|
|
|
## RULE: Show Model Download Progress
|
|
|
|
**Status**: MANDATORY
|
|
|
|
### Rule Definition
|
|
- **ALWAYS** show download progress when loading Whisper models
|
|
- Display percentage and/or file size being downloaded
|
|
- Models can be 150MB-500MB+ and take significant time to download
|
|
|
|
### Why This Rule Exists
|
|
1. **User Experience**: Users need to know the app isn't frozen
|
|
2. **Transparency**: Large downloads should be visible
|
|
3. **Expectations**: Users can estimate wait time
|
|
|
|
### Implementation Requirements
|
|
- Use progress callback from transformers.js pipeline
|
|
- Display progress percentage in status element
|
|
- Show "Downloading model: X%" or similar message
|
|
|