Skip to content

Add seated PSVR webcam 6DoF tracking - #60

Draft
Davidbkr03 wants to merge 1 commit into
Timocop:master-experimentalfrom
Davidbkr03:codex/psvr-headset-led-6dof
Draft

Add seated PSVR webcam 6DoF tracking#60
Davidbkr03 wants to merge 1 commit into
Timocop:master-experimentalfrom
Davidbkr03:codex/psvr-headset-led-6dof

Conversation

@Davidbkr03

Copy link
Copy Markdown

Summary

  • Add an opt-in generic webcam tracker backend using Windows Media Foundation, stable device identities, exact native mode persistence, bounded latest-frame capture, and camera-control readback.
  • Track the PSVR headset's built-in LED constellation with a deterministic correspondence/PnP solver, geometric quality gates, and recovery priors suitable for a seated, single-camera setup.
  • Fuse optical translation and capture-time optical orientation with timestamped Morpheus IMU orientation so rendering keeps the headset's low-latency rotational updates while the camera corrects 6DoF position.
  • Add configuration/protocol support, calibration-mode gating, config migration, operational diagnostics, deterministic tests, and a complete setup/hardware-validation guide.
  • Preserve the existing PS3 Eye/controller tracking path; generic webcam discovery is allowlisted and disabled by default.

Reason

PSMoveServiceEx can read the PSVR headset sensors, but it did not provide a production path for tracking the headset's built-in LEDs through a normal webcam. This change supplies the missing optical position path and aligns it with the headset's own clocked IMU data for seated PC VR use.

Implementation

  • Windows Media Foundation enumeration and asynchronous Source Reader capture.
  • Stable symbolic-link-derived tracker IDs and explicit allowlisting.
  • Exact video/calibration mode identity, progressive-mode selection, stride-aware BGR/MJPEG conversion, and normalized exposure/gain controls.
  • Versioned nine-point PSVR LED model, configurable front-LED mask/intensity, contour extraction, bounded correspondence search, Hungarian assignment, PnP refinement, and reprojection/depth/geometry gates.
  • 24-bit 1 MHz Morpheus sensor-clock unwrap and host-clock synchronization.
  • Delayed-frame fusion using capture-time IMU alignment, current-IMU orientation advancement, position age compensation, duplicate/stale/future rejection, and clear tracking status diagnostics.
  • Config Tool and protocol extensions with older-client/config compatibility.

Testing performed

  • Fresh strict Linux build with GCC 13, CMake 3.28, OpenCV 4.6, and -Wall -Wextra -Werror -pedantic.
  • ctest: 5/5 tests passed:
    • Morpheus sensor clock
    • point-cloud pose solver
    • optical/IMU fusion
    • video-source helpers
    • generic-webcam enumerator helpers
  • Generated and compiled PSMoveProtocol.
  • Compiled the feature-facing service objects available in the Linux environment, including the solver/fusion, HMD/tracker views and managers, Morpheus device path, PS3Eye tracker, generic enumerator, and non-Windows video-source path.
  • Independent static audits found no remaining code-proven P0/P1 defects.

The full Linux service target remains blocked by an existing upstream/pinned-submodule mismatch: ControllerManager.cpp calls Gamepad_init(gamepad_api_xinput_only), while the pinned libstem exposes Gamepad_init() with no argument. That unrelated semantic mismatch was not papered over in this PR.

Documentation updated

  • Add misc/PSVR_LED_TRACKING.md with architecture, configuration, calibration, diagnostics, test coverage, hardware acceptance gates, rollback, and iVRy handoff.
  • Link the guide from README.md.

Remaining validation / why this is a draft

  • Build the Windows-only Media Foundation and Config Tool paths with the repository's supported MSVC/Windows SDK toolchain.
  • Run physical calibration and validate LED coordinates, camera intrinsics/distortion, PSVR IMU-to-optical axes, exposure/gain behavior, capture latency, occlusion recovery, and seated tracking volume.
  • Verify the final pose is consumed correctly by the target iVRy/SteamVR runtime.

No PSVR, webcam, native Windows toolchain, or iVRy runtime was available for this implementation pass, so those hardware/runtime checks are intentionally not claimed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant