feat(video): extract what is on screen, aligned with what was said - #11
Merged
Merged
Conversation
webindex video frames takes a frame at every scene change (ffmpeg scene score above 0.3) and just after every chapter start, evenly spaced ones for a video with neither, drops near-duplicates by dHash (64 bits, Hamming <= 6, no dependency: ffmpeg shrinks each frame to 9x8 grey) and keeps at most 20/50/100 by --effort, the most widely spaced first. The frames land in <id>/frames/NNNN_mm-ss.jpg, and FRAMES.md pairs each with the transcript from 5 s before it to 10 s after. The video itself lives in a temp directory removed afterwards. Refs #7
|
🎉 This PR is included in version 1.24.0 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Third of four PRs for #7.
webindex video frames <url|id|dir> [--effort low|med|high]:select='gt(scene,0.3)',showinfo, timestamps read back frompts_time) and just after every chapter start, and evenly spaced ones for a video with neither;--effort(medby default), the most widely spaced first, chapter frames last to go;<id>/frames/NNNN_mm-ss.jpg,FRAMES.md(each frame with the transcript from t−5 s to t+10 s, a long passage quoted once) andframes.json.Library:
extractFrames,FRAME_EFFORT.Checked live on two code tutorials (Fireship "TypeScript in 100 Seconds", "React in 100 Seconds"): the frame at 00:42 shows
Cannot find name 'someObject'in the IDE next to "the ide is providing feedback about using a variable that doesn't exist"; the one at 00:51 showstsc index.tsnext to "the tsc command".