Gan, Jimmy
784a67c921
- Update index.html file input to accept audio/*,video/* - Add engine selection dropdown (Local vs Server) in index.html - Implement JS bufferToWav WAV encoder in app.js (Float32Array PCM to 16-bit WAV) - Integrate server-side transcribing route via FormData POST in app.js - Update DEVELOPMENT_RULES.md to mark browser-side extraction as implemented Co-Authored-By: Claude Opus 4.8 <noreply@anthreply.com>
2.1 KiB
2.1 KiB
Media to Text - Development Rules
CRITICAL RULE: Never Upload Full Video Files
Status: MANDATORY - NEVER VIOLATE
Rule Definition
- NEVER allow users to upload full video files (MP4, AVI, MOV, MKV, FLV, WMV, WEBM) to the server
- ALWAYS extract audio on the browser BEFORE uploading
- Only audio files (MP3, WAV, M4A, etc.) should be uploaded to the server
Why This Rule Exists
- Bandwidth Efficiency: Video files are 10-100x larger than extracted audio
- User Experience: Faster uploads = better UX
- Server Resources: Reduces server storage and processing load
- Cost: Minimizes data transfer costs
Implementation Requirements
- Browser-side audio extraction is MANDATORY for video files
- Use ffmpeg.wasm or equivalent WebAssembly solution
- If browser extraction fails, show error message - DO NOT fall back to uploading full video
- Server-side extraction is NOT acceptable as a workaround
Current Status
- ✅ Browser extraction via Web Audio API (
AudioContext.decodeAudioData): IMPLEMENTED- Converts media file data (audio or video) into a
Float32ArrayPCM buffer client-side in the browser. - For server upload, encodes the PCM buffer to a standard 16-bit mono 16kHz WAV Blob in Javascript.
- Ensures absolute compliance with the rule that full video files are never uploaded to the server.
- Converts media file data (audio or video) into a
NOT Acceptable
- ❌ Uploading full video files to server
- ❌ Server-side extraction as primary solution
- ❌ Removing this rule without explicit approval
RULE: Show Model Download Progress
Status: MANDATORY
Rule Definition
- ALWAYS show download progress when loading Whisper models
- Display percentage and/or file size being downloaded
- Models can be 150MB-500MB+ and take significant time to download
Why This Rule Exists
- User Experience: Users need to know the app isn't frozen
- Transparency: Large downloads should be visible
- Expectations: Users can estimate wait time
Implementation Requirements
- Use progress callback from transformers.js pipeline
- Display progress percentage in status element
- Show "Downloading model: X%" or similar message