Skip to content

Keep client study data out of the public repo and add a data manifest #227

Closed

Description

@Sanjana-Siddani

Is your feature request related to a problem? Please describe.
Dr. Gajapriya sent 19 telemetry files from a human-participant study. The repo is public, and inference/Data/ has held committed CSVs before. Nothing stops someone from committing these files by accident.

Describe the solution you'd like

  • Add inference/Data/client/ to .gitignore, along with every generated CSV (*_features.csv, pilot_clusters.csv, group_centres.csv). The second half was also raised in the 215 deviation features #218 review.
  • Commit inference/Data/client/MANIFEST.md. It lists file names, the pilot/trial index, row counts, a SHA-256 per file and where the team can get the files (a shared drive), and contains no data.
  • Ask Dr. Gajapriya whether derived per-flight features can be committed, and whether there is an IRB or data-use condition we need to follow.

Acceptance criteria

  • git status stays clean after copying the files in and running the whole pipeline
  • The manifest lists all 19 files with their hashes
  • The client's answer on sharing derived features is recorded in this issue

Activity

  1. added
    documentationImprovements or additions to documentation
    area:mlML / inference work
    needs-infoWaiting on reporter for more information
    on Oct 12, 2026
  2. jmacd867 commented on Oct 15, 2026

    @jmacd867
    Collaborator

    One note before this lands:

    • The .gitignore half doesn't depend on the client's answer and can land now. The data-handling questions going to the client at the upcoming meeting: is there an IRB protocol or data-use agreement, what does it require (storage, access, deletion), are students covered by it or do we need to be added and complete human-subjects training, and may derived data (per-flight features, group assignments) be committed.
  3. self-assigned this
    on Oct 15, 2026
  4. linked a pull request that will close this issueMLmodel / clustering #225on Oct 15, 2026
  5. jmacd867 commented on Oct 16, 2026

    @jmacd867
    Collaborator

    Answers from the client meeting (`docs/client_meeting_notes.md`):

    • There is an IRB protocol. Only people named on it can access the full data (the client holds it, so the sample is limited). Copies of the full data may not be made or published.
    • The files we received are de-identified, and using them in this class project is fine.
    • Publishing results (journal, conference proceedings) is allowed.
    • The client asked whether the repo is public (it is) and confirmed we will not upload raw files.

    Still not answered: whether derived per-flight features can be committed, whether the students must be added to the protocol or complete human-subjects training, and storage, access and deletion rules. The `.gitignore` half can land now.

  6. added this to the v1.0 milestone on Oct 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:mlML / inference workdocumentationImprovements or additions to documentationneeds-infoWaiting on reporter for more information

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions