Add a feature-engineering step that computes the four deviation metrics #215

Activity

1. Phlabry

   assigned

   jmacd867

   on Sep 28, 2026
2. Phlabry

   added

   enhancementNew feature or requestNew feature or request

   python

   on Sep 28, 2026
3. jmacd867 commented on Oct 3, 2026

   

   jmacd867

   on Oct 3, 2026

   Collaborator

   More actions

   • Naming: use glossary terms. New columns are airspeed_deviation and target_airspeed. The CSV column stays velocity and is mapped at the loading boundary. Update docs/telemetry_schema.md §4a and §5 to match.
   • Input: a table with pilot_id, flight_id, the four actuals and four target_* columns, one row per sample. Targets may be constant or vary within a flight. A missing column raises an error that names it. Until Record pilot_id and flight_id in the logged CSV #211 and Capture designated (target) values for altitude, heading, vertical speed and airspeed #213 land, only synthetic data can supply this.
   • Layout: a small shared module for the wrapped angular difference, used by both features.py and label_generator.py. features.py holds pure DataFrame-in/DataFrame-out functions (row deviations, then per-flight and per-pilot aggregates). A thin CLI reads and writes CSVs.
   • Heading helper range: [-180, 180), as in the schema doc. This changes label_generator.py at exactly 180 degrees: _calculate_heading_change returns +180 today (read as a right turn at line 141) and would return -180 (a left turn at line 137). This is a one-sample, float-exact case. Note it in the PR.
   • Aggregates per metric: mean absolute deviation and standard deviation of signed deviation. Also emit mean signed deviation (bias), but leave it out of the default clustering set. No time-outside-tolerance until the client supplies thresholds.
   • Roll-up: aggregate each flight first, then average the flight features per pilot, so every flight counts equally. Keep the per-flight table as an output.
   • Scaling: Add a feature-engineering step that computes the four deviation metrics #215 outputs raw-unit aggregates and documents the units. Cross-pilot scaling belongs in Build a baseline pilot clustering model #216. Do not z-score per pilot, because that erases the between-pilot differences we are clustering on.
   • Missing targets: deviation is NaN where the target is NaN. Aggregates skip those rows and record rows-used per flight. A flight with zero valid rows raises an error, so no pilot silently gets all-NaN features.
   • Legacy data: do not rescale inside this step (Committed flight CSVs still hold the pre-#196 altitude and velocity scaling #212 owns that). The module docstring states that input must be post-Plugin applies metric conversion factors to pilot altitude and airspeed that are already in feet and knots #196 units.
   • Tests: add pytest as a dev dependency in inference/. Cover the angle helper (wrap at 360, exactly 180) and the aggregates (NaN targets, an empty flight), using synthetic targets.
4. jmacd867

   marked this as blocking on Oct 3, 2026

   • Build a baseline pilot clustering model #216
5. jmacd867

   added

   area:mlML / inference workML / inference work

   on Oct 3, 2026
6. Phlabry commented on Oct 5, 2026

   

   Phlabry

   on Oct 5, 2026

   CollaboratorAuthor

   More actions

   @jmacd867 #219 adds pilot_id and flight_id to the logged CSV, and they pass through label_generator into labeled_flight_data.csv. That is the #211 half of your input schema. Targets still wait on #213.
7. jmacd867

   closed this as completedin #218on Oct 7, 2026
8. PRINCEKK122

   added 2 commits that reference this issue on Oct 15, 2026

   Add deviation features for pilot clustering (oss-slu#215)

   ...

   488424a

   Closes oss-slu#215

   be914c8