Skip to content

Keystroke overlay: show the shortcuts you press in the video #2

Description

@nftechie

Context

Tutorials and bug reports are easier to follow when viewers can see "⌘K" or "⇧⌘P" as you press it. Snapcap has no input overlays today.

Anything Snapcap draws on the video follows one rule: record data, render at export. Camera bubble drags are saved as CameraMoves in project.json and animated by the exporter, so they can be edited or switched off later. Keystrokes should work the same way, not be burned into screen.mov.

Where to look

  • Sources/SnapcapCore/Capture/RecordingSession.swift: recordCameraMove(at:x:y:) stamps events with elapsed (recording time, pauses excluded) and finish(overlay:) saves them.
  • Sources/SnapcapCore/Model/CameraOverlay.swift and Sources/SnapcapCore/Model/Project.swift: lenient init(from:) decoding, and renderFingerprint's Inputs.
  • Sources/SnapcapCore/Render/FrameComposer.swift: draws each frame; time is recording time.
  • Sources/SnapcapCore/Render/OverlayCompositor.swift: passes compositionTime + instruction.timeOffset (the trim start) to the composer.
  • Sources/SnapcapCore/Render/OverlayChrome.swift: the pattern for drawing with Core Graphics once and caching the result as a CIImage.
  • Sources/Snapcap/Views/EditorView.swift: PlayerStage layers live overlays over the player at playback.currentTime.
  • Sources/Snapcap/App/AppModel.swift: the recording lifecycle and permission patterns.
  • docs/ARCHITECTURE.md, especially "Common changes → Add a timed overlay".

Proposal

  1. Capture. Add an opt-in "Show keystrokes" setting. While recording, listen for key events with a listen-only CGEventTap or NSEvent.addGlobalMonitorForEvents. These need the Input Monitoring or Accessibility permission, so explain why and request it only when the toggle is turned on.
  2. Privacy by default. Record only combinations that include ⌘, ⌃ or ⌥, plus a few named keys (Esc, Tab, Return, arrows), unless the user opts into all keys. Password fields enable Secure Event Input, which blocks taps; handle that quietly.
  3. Model. Something like "keystrokes": { "isVisible": true, "position": "bottomCenter", "events": [{ "time": 3.2, "keys": "⇧⌘P" }] }, decoded leniently and added to the fingerprint.
  4. Render. Draw a rounded keycap pill for about 1.2 s after each event, merging rapid repeats. Size it in the same 1080-reference units as the bubble (OverlayLayout.unitScale), in both FrameComposer and the editor preview.
  5. Edit. Add an editor toggle and a position picker.

Splitting this into PRs is welcome: capture, then model and render, then the editor. Please comment with the proposed JSON shape before starting.

Acceptance criteria

  • Off by default. Turning it on explains and requests permission, and recording still works if permission is denied.
  • Events are stored in project.json in recording time, and older projects without the key still open.
  • The export and the editor preview show the same overlay at the same times, including in trimmed exports.
  • Plain typing isn't recorded unless the user opts in.
  • Tests cover decoding, plus a fixture-based export test that finds overlay pixels at an event time.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    captureScreen, camera and audio captureenhancementNew feature or requestexportRendering and exporting videohelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions