From 427ee4faf7af08295fc5d626f1c5c66dbc6e2dee Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 06:55:03 +0000 Subject: [PATCH 1/2] feat(readers): several animals in one Anipose file Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01N7hYfMaQFqSMBtq9ZWJ3gV --- CHANGELOG.md | 1 + README.md | 2 +- .../2026-10-10-native-3d-files-design.md | 9 +- docs/INTEROPERABILITY.md | 2 +- docs/METRICS_SPEC.md | 2 +- docs/dev/DECISIONS.md | 8 ++ docs/guide/USER_GUIDE.md | 4 +- docs/wiki/Compatibility.md | 2 +- tests/test_integration/test_native_3d.py | 35 +++++ tests/test_native3d/test_gating.py | 24 ++++ tests/test_native3d/test_guards.py | 70 ++++++++-- tests/test_readers/native3d_files.py | 25 +++- tests/test_readers/test_anipose.py | 95 ++++++++++++++ track2data/api.py | 6 +- track2data/core/models.py | 8 +- track2data/metrics/availability.py | 12 +- track2data/preprocess/identity_switch.py | 21 +-- track2data/preprocess/pipeline.py | 6 +- track2data/readers/anipose.py | 123 ++++++++++++++++-- track2data/readers/native3d.py | 16 +++ 20 files changed, 413 insertions(+), 58 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7613160..9a4866d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ### Added +- **Several animals in one Anipose file.** List the animals' column prefixes in the new **Animal column prefixes** option (`--option animals=fishA,fishB`) and the file's `fishA__x|y|z` and `fishB_...` columns become two animals of one session, in that order. Group metrics are then offered for the project when every native file holds several animals. Identity-switch correction now relabels a native file's vertical coordinate together with x and y. Files without the option read exactly as before (decision D-053). - **Three more 3-D metrics.** IL-18 *Vertical Speed* (mean, median and maximum of |dz/dt| in cm/s, plus body lengths per second), GL-18 *3-D Inter-Individual Distance* (mean over all pairs, the 3-D counterpart of GL-2) and GL-19 *3-D Group Cohesion* (reciprocal of the 3-D nearest-neighbour or inter-individual distance, chosen by `cohesion_source`). They need a fused or native 3-D session and a depth scale, skip frames with a missing animal as GL-16 does, and are greyed out otherwise. The registry now holds 57 metrics (decision D-052). - **Warning before a run when most animal slots are empty.** A multi-animal session in which at least half the (frame, animal) slots have no position, typically a Ctrax file with one slot per track fragment, now raises an advisory in `track2data validate`, the log and the run README, saying that group metrics skip any frame with a missing animal and pointing to the `top_n` option. - **Metadata: Excel workbooks with several sheets.** The metadata screen loads `.xlsx` files (it only took CSV before) and, when the workbook has more than one sheet, shows a **Sheet** menu; the choice is saved with the project. Projects without a chosen sheet are unchanged, including their project hash. The old `.xls` format is refused with a message that says to save as `.xlsx` or CSV. diff --git a/README.md b/README.md index 119451d..14f8ff7 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ It is a desktop application and a Python engine you can drive headlessly for bat **Reads** idtracker.ai (6.x and v5), DeepLabCut (CSV, also Lightning Pose), SLEAP (analysis HDF5), Ctrax (raw `.mat`), `trx.mat`, and two native 3-D formats, Anipose `pose-3d` CSV and DeepLabCut 3-D tables (positions in physical units; the 3-D metrics -the 3-D metrics IL-16 and IL-17 run on them, and IL-15 once you give the water column; GL-16 does not, because a native file holds one animal). Run +the 3-D metrics IL-16 and IL-17 run on them, and IL-15 once you give the water column; GL-16 does not, because a native file holds one animal unless you list the animals of an Anipose file). Run `track2data list-readers` for the list and the options each reader needs. **Individual** โ€” distance travelled, speed, acceleration, tortuosity, diff --git a/docs/3d-movement/2026-10-10-native-3d-files-design.md b/docs/3d-movement/2026-10-10-native-3d-files-design.md index c64ba60..57c286b 100644 --- a/docs/3d-movement/2026-10-10-native-3d-files-design.md +++ b/docs/3d-movement/2026-10-10-native-3d-files-design.md @@ -23,8 +23,11 @@ A adds readers for both and lets the 3-D metrics of B2 and IL-15 run on them. `anipose` (pose-3d CSV) and `deeplabcut_3d`. There are no real sample files at hand for the DLC-3D layout. For Anipose the repo's published sample (`tests/real_samples/`) is used by the existing reader-contract case when the fixture is present; it is not required. -- **One animal per file.** Neither format carries an animal axis. Several animals are several files - (several sessions), as for the other single-animal readers. +- **One animal per file, unless the user lists the animals (Anipose, D-053).** Neither format carries + an animal axis. Several animals are several files (several sessions), as for the other + single-animal readers. An Anipose file can hold several animals as column prefixes + (`__x|y|z`) when the `animals` option names them in order; the prefixes are + never guessed, since `left_ear` / `right_ear` look the same as `fishA_nose` / `fishB_nose`. - **Units are converted when reading, to the project's length unit.** The reader takes `unit` (the file's unit: mm, cm, m, in; Anipose proposes mm, which the file does not state, so it is always confirmed) and `target_unit` (default cm). The first import sets the project's @@ -139,7 +142,7 @@ code does. - **IL-17 reads `z`.** The first build left IL-17 on `depth * height`, so a native session without the water column got NaN; found by the end-to-end test and fixed. IL-15 outside values, and non-finite z, are NaN and counted. -- **GL-16 is not offered for native files** (one animal per file; the metric needs two in a session). +- **GL-16 is not offered for native files** unless every native file of the project lists several animals (the metric needs two in a session). - **The unreadable-format notice for DLC-3D was narrowed** in `readers/recognise.py`: the single-animal tables the reader takes are no longer reported, while a multi-animal table (extra `individuals` header row) or one without full x, y, z triples is still named as one that cannot be read yet. diff --git a/docs/INTEROPERABILITY.md b/docs/INTEROPERABILITY.md index 14ba077..3025a14 100644 --- a/docs/INTEROPERABILITY.md +++ b/docs/INTEROPERABILITY.md @@ -110,7 +110,7 @@ rides along as an attribute. | Ctrax raw `.mat` input | โœ… supported | The file Ctrax's *Save Tracks as Matlab File* writes. Frame size is an option you give; the frame rate comes from the file. Ctrax track ids are fragments, so a session is identity-free and one slot per id (`top_n` keeps the longest). See `docs/tracker-formats/README.md`. | | `trx.mat` input (Ctrax / FlyTracker / JAABA) | ๐Ÿงช synthetic only | A classic (v5/v7) MATLAB file holding a `trx` struct, built from the documented layout with no real sample pinned yet. Frame size is an option; `fps` is read from the file. `pxpermm` (pixels per mm) becomes the session scale for Session calibration, pending the usual unit confirmation; a value of exactly 1 is ignored. v7.3 files are not read. | | DeepLabCut `.h5` input | ๐Ÿงช synthetic only | The `.h5` DeepLabCut writes beside the CSV (pandas table under `df_with_missing`). Read with h5py; the pickled column labels are decoded by an unpickler that refuses every global, and PyTables is not needed (D-050). The frame rate and frame size are options you give; the keypoint is typed, not picked from a list. A CSV of the same name beside it is the one session. Multi-animal tracklet files (`*_el.h5`) are refused. Becomes *supported* once the real EPM sample has been read in a run. | -| Anipose pose-3d CSV input | ๐Ÿงช synthetic only | One animal per file, native 3-D (X, Y, Z in the file's unit). Built from the documented layout; no real sample pinned yet. Several animals in one file are not read. | +| Anipose pose-3d CSV input | ๐Ÿงช synthetic only | Native 3-D (X, Y, Z in the file's unit). One animal per file, or several when the `animals` option lists their column prefixes (`__x|y|z`). Built from the documented layout; no real sample pinned yet. | | DeepLabCut-3D input | ๐Ÿงช synthetic only | The triangulated 3-D table DeepLabCut writes, read as native 3-D. Built from the documented layout; no real sample pinned yet. | | NWB | โŒ not planned | Would be worth revisiting if the behavioural-NWB extensions stabilise. | diff --git a/docs/METRICS_SPEC.md b/docs/METRICS_SPEC.md index 7428f18..bd3f503 100644 --- a/docs/METRICS_SPEC.md +++ b/docs/METRICS_SPEC.md @@ -631,7 +631,7 @@ info-button modal (ยง6). ### 4.3 Social spacing -**Availability (native 3-D projects).** Every group metric (`GL-*`) is unavailable in a native 3-D project (Anipose, DeepLabCut 3-D files): "needs several animals in one session; a native 3-D file holds one animal". The Metrics screen greys them out with that reason, and `validate` and the run `README.md` list it. +**Availability (native 3-D projects).** Every group metric (`GL-*`) is unavailable in a native 3-D project (Anipose, DeepLabCut 3-D files) unless every file holds several animals (an Anipose file with its `animals` option set); otherwise: "needs several animals in one session; a native 3-D file holds one animal". The Metrics screen greys them out with that reason, and `validate` and the run `README.md` list it. #### GL-1 โ€” Nearest-neighbour distance (NND) diff --git a/docs/dev/DECISIONS.md b/docs/dev/DECISIONS.md index c449439..9455f7f 100644 --- a/docs/dev/DECISIONS.md +++ b/docs/dev/DECISIONS.md @@ -1256,3 +1256,11 @@ could not be re-added, while the store would have accepted it. **Why:** one frame rule and one body-length rule keep the three group metrics comparable, and the NND default keeps GL-19 the direct 3-D analogue of GL-6. +### D-053 ยท Several animals in one Anipose file: a column prefix per animal, named by the user + +**Status:** accepted (the user chose option (a)); implemented. Synthetic files only, like the rest of the reader. + +**Decision:** the Anipose reader takes an `animals` option, a comma-separated list of column prefixes (`fishA,fishB`). A keypoint column belongs to the longest listed prefix it starts with (`__x|y|z`); the list order is the slot order and the labels are stored as `identities_labels`. Every animal must have the same keypoints and every keypoint column must belong to a listed animal, else `ANIPOSE_ANIMAL_COLUMNS`. The option left empty reads one animal, exactly as before. Group metrics are offered for a native project when every native session lists two or more animals (`manifest_native_several_animals`); for a run, the preprocessed session's animal count decides. Identity-switch correction reports the permutation it applied (`PPStepResult.permutation`) and the pipeline applies it to the native z, so z stays with its animal. + +**Why:** the prefixes are not guessed because a single-animal file with `left_ear` and `right_ear` has the same shape as two animals with one keypoint each; guessing would turn a one-animal file into two. A project with one single-animal native file keeps the group metrics off, since a group metric on a one-animal session has nothing to compare. + diff --git a/docs/guide/USER_GUIDE.md b/docs/guide/USER_GUIDE.md index 18e251b..ae0d9ba 100644 --- a/docs/guide/USER_GUIDE.md +++ b/docs/guide/USER_GUIDE.md @@ -788,14 +788,14 @@ no fusion): the positions are `(x, y, z)` in the project's unit, and IL-16 and I column. Example: a fish that moves (3, 4, 12) mm per frame in an Anipose file at 25 frames per second, read as cm, steps hypot(0.3, 0.4, 1.2) = 1.3 cm per frame, so 10 steps give an IL-16 of 13 cm and an IL-17 of 1.3 x 25 = 32.5 cm/s; its 2-D metrics measure the horizontal steps of 0.5 cm (IL-1 5 cm, IL-2 -12.5 cm/s). **GL-16 is not offered for native files** (it needs two animals in one session and a file holds one), and neither is any other group metric ("needs several animals in one session; a native 3-D file holds one animal"). **IL-3, IL-9, IL-10 and IL-14 are not offered either** ("uses a pixel-sized default and the project's native 3-D files have no pixel frame"): their default thresholds are in pixels and would silently change with the project unit. The Metrics screen greys them out, `validate` and the run `README.md` list the reason. IL-15 appears once the water column is set. +12.5 cm/s). **GL-16 is not offered for native files that hold one animal** (it needs two animals in one session), and neither is any other group metric ("needs several animals in one session; a native 3-D file holds one animal"). An Anipose file that holds several animals, with columns named `__x`, `_y`, `_z`, offers them when you list the prefixes in the **Animal column prefixes** option (`fishA,fishB`, in the order you want the animals); every native file in the project must then do the same. **IL-3, IL-9, IL-10 and IL-14 are not offered either** ("uses a pixel-sized default and the project's native 3-D files have no pixel frame"): their default thresholds are in pixels and would silently change with the project unit. The Metrics screen greys them out, `validate` and the run `README.md` list the reason. IL-15 appears once the water column is set. The per-frame table has a `z_cm` column, the file's own vertical coordinate after the same gap filling and smoothing as x and y (so IL-16 can be reproduced from `x_cm`, `y_cm` and `z_cm`); with a water column it also carries `depth_fraction` and `depth_cm` as for a fused unit. The run's `README.md` has a **Native 3-D source** section (reader, file unit, project unit, vertical axis, flip, water column or its absence) and, with a water column, a row **Samples outside the water column** with the count left out. -Limits of native 3-D files: one animal per file; no mixing with pixel sessions; the unit you give +Limits of native 3-D files: one animal per file (several only for an Anipose file whose animal prefixes you list); no mixing with pixel sessions; the unit you give must be the one of the calibration board (a wrong one scales everything); no refraction correction; the water column is entered, not estimated; no zones or background image; no group metrics and no IL-3, IL-9, IL-10 or IL-14; a multi-animal DeepLabCut 3-D table (with an `individuals` header row) is named by the scan as one it cannot read yet. diff --git a/docs/wiki/Compatibility.md b/docs/wiki/Compatibility.md index 454780b..b7e51f9 100644 --- a/docs/wiki/Compatibility.md +++ b/docs/wiki/Compatibility.md @@ -14,7 +14,7 @@ is authoritative. | **DeepLabCut** (CSV; also Lightning Pose) | **Supported** | One session per file. Frame rate and frame size are required options; the optional **Scale (pixels per cm)** feeds Session calibration | | **SLEAP** (analysis HDF5) | **Supported** | Export Analysis HDF5 from SLEAP. Same required options and optional scale as DeepLabCut | | **Ctrax** (raw `.mat`) | **Supported** | Frame size is a required option; the frame rate comes from the file. Ctrax track ids are fragments, so a session is identity-free. Optional scale as above | -| **Anipose** (`pose-3d` CSV) | **Unverified** | Built from the published layout, tested on synthetic files only. Native 3-D: one animal per file; the frame rate and the unit of the coordinates are required options (never guessed). See the 3-D section of the user guide | +| **Anipose** (`pose-3d` CSV) | **Unverified** | Built from the published layout, tested on synthetic files only. Native 3-D: one animal per file, or several when you list their column prefixes in the `animals` option; the frame rate and the unit of the coordinates are required options (never guessed). See the 3-D section of the user guide | | **DeepLabCut 3-D** (triangulated CSV) | **Unverified** | Same as Anipose; the unit is the calibration board's and is not proposed. Multi-animal 3-D tables are not read (the scan names them) | | **Ctrax / FlyTracker / JAABA** (`trx.mat`) | **Unverified** | Built from the documented layout with no real sample yet, and labelled as such in the confirm step. Classic MATLAB files only. `pxpermm` becomes the session scale | diff --git a/tests/test_integration/test_native_3d.py b/tests/test_integration/test_native_3d.py index e66bbd2..7360753 100644 --- a/tests/test_integration/test_native_3d.py +++ b/tests/test_integration/test_native_3d.py @@ -350,3 +350,38 @@ def test_il15_names_the_water_column_as_its_source(project: Path) -> None: psess = engine.preprocess(engine.import_ref(manifest.sessions[0])) result = engine.compute_metrics(psess)["IL-15"] assert set(result["depth_extent_source"]) == {"water_column"} + + +# โ”€โ”€ one file, several animals (prefix per animal) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +def test_two_animals_in_one_file_run_the_group_metrics(tmp_path: Path) -> None: + """Two fish 10 cm apart in x (100 mm), moving together: GL-16 sees them, IL-16 sees each.""" + data = tmp_path / "data" + data.mkdir() + xyz = np.stack([_track(), _track() + np.array([100.0, 0.0, 0.0])], axis=1) # (F, 2, 1, 3) + write_anipose_csv( + data / "pair.csv", ["head"], N_FRAMES, animals=["f1", "f2"], xyz=xyz + ) + now = datetime.now(tz=UTC) + project = tmp_path / "p.t2d.json" + write_manifest(ProjectManifest(project_name="pair", created_at=now, updated_at=now), project) + result = CliRunner().invoke( + cli, + ["add", str(project), str(data), "--option", "unit=mm", "--option", "fps=25", + "--option", "animals=f1,f2", "--yes"], + ) + assert result.exit_code == 0, result.output + manifest = read_manifest(project) + manifest = manifest.model_copy( + update={ + "metrics": SELECTION.model_copy(update={"individual": ["IL-16"], "group": ["GL-16"]}) + } + ) + out = tmp_path / "out" + Engine(manifest).run(out, exporters=["csv_long"]) + long = pd.read_csv(out / "pair" / "metrics_long.csv") + path = long[(long["metric_id"] == "IL-16") & (long["column"] == "path_length_3d_cm")] + assert sorted(path["individual_id"].tolist()) == [0, 1] + assert path["value"].tolist() == pytest.approx([PATH_CM, PATH_CM], abs=1e-9) + assert _value(long, "GL-16", "mean_nnd_3d_cm") == pytest.approx(10.0, abs=1e-9) diff --git a/tests/test_native3d/test_gating.py b/tests/test_native3d/test_gating.py index 26fa7b6..736c8d3 100644 --- a/tests/test_native3d/test_gating.py +++ b/tests/test_native3d/test_gating.py @@ -71,3 +71,27 @@ def test_pixel_and_fused_projects_still_offer_them() -> None: skipped = Engine(fused).skipped_metrics(False) reasons = {NEEDS_PIXEL_FRAME_REASON, NEEDS_SEVERAL_ANIMALS_REASON} assert not reasons & set(skipped.values()) + + +def _pair_ref(session_id: str): + ref = _ref(session_id) + return ref.model_copy(update={"reader_options": {**ref.reader_options, "animals": "a, b"}}) + + +def _group_reasons(refs) -> dict[str, str]: + manifest = _manifest(refs, calibration=SESSION_CM, individual=STILL_OFFERED, group=GROUP) + return {m: r for m, r in Engine(manifest).skipped_metrics(False).items() if m in GROUP} + + +def test_group_metrics_are_offered_for_a_native_session_with_several_animals() -> None: + assert _group_reasons([_pair_ref("n1")]) == {} + + +def test_group_metrics_stay_unavailable_for_a_one_animal_native_session() -> None: + reasons = _group_reasons([_ref("n1")]) + assert set(reasons) == set(GROUP) and set(reasons.values()) == {NEEDS_SEVERAL_ANIMALS_REASON} + + +def test_one_single_animal_file_rules_the_group_metrics_out_for_the_project() -> None: + reasons = _group_reasons([_pair_ref("n1"), _ref("n2")]) + assert set(reasons) == set(GROUP) diff --git a/tests/test_native3d/test_guards.py b/tests/test_native3d/test_guards.py index 5abb7fc..8a95df4 100644 --- a/tests/test_native3d/test_guards.py +++ b/tests/test_native3d/test_guards.py @@ -69,19 +69,67 @@ def test_fuse_refusal_names_the_missing_pixel_frame() -> None: ) -def test_switch_correction_warns_for_multi_animal_z(caplog) -> None: +def _crossing_pair(n: int = 40, swap_at: int = 20): + """Two fish on straight parallel lines whose labels swap at *swap_at*; z tells them apart.""" + t = np.arange(n, dtype=float) + xy = np.zeros((n, 2, 2)) + xy[:, 0, 0], xy[:, 0, 1] = t, 0.0 + xy[:, 1, 0], xy[:, 1, 1] = t, 30.0 + z = np.zeros((n, 2)) + z[:, 0], z[:, 1] = 5.0, 50.0 + swapped = xy.copy() + swapped_z = z.copy() + swapped[swap_at:] = xy[swap_at:, ::-1] + swapped_z[swap_at:] = z[swap_at:, ::-1] + return xy, z, swapped, swapped_z + + +def test_switch_correction_reports_which_column_holds_each_identity() -> None: from track2data.preprocess.identity_switch import correct_switches - xy = np.random.default_rng(2).random((10, 2, 2)) * 50 - cfg = IdSwitchCfg(enabled=True) - with caplog.at_level(logging.WARNING, logger="track2data.preprocess.identity_switch"): - correct_switches(xy, cfg, has_z=True) - z_warnings = [r for r in caplog.records if "3-D z coordinate" in r.getMessage()] - assert len(z_warnings) == 1 - caplog.clear() - with caplog.at_level(logging.WARNING, logger="track2data.preprocess.identity_switch"): - correct_switches(xy, cfg) - assert not [r for r in caplog.records if "3-D z coordinate" in r.getMessage()] + xy, _, swapped, _ = _crossing_pair() + fixed, step = correct_switches(swapped, IdSwitchCfg(enabled=True)) + np.testing.assert_allclose(fixed, xy) + assert step.permutation is not None and step.permutation.shape == (40, 2) + assert step.permutation[:20].tolist() == [[0, 1]] * 20 + assert step.permutation[20:].tolist() == [[1, 0]] * 20 + + +def test_identity_switch_correction_leaves_native_z_aligned_with_xy() -> None: + """A swapped label moves z with x and y: the animal keeps its own height.""" + from track2data.core.models import PreprocessConfig + from track2data.preprocess.pipeline import run + + xy, z, swapped, swapped_z = _crossing_pair() + session = _native_session(swapped, swapped_z) + cfg = PreprocessConfig() + cfg.identity_switch = IdSwitchCfg(enabled=True) + cfg.smoothing.enabled = False + cfg.gap_fill.enabled = False + psess = run(session, cfg) + np.testing.assert_allclose(psess.xy, xy) + np.testing.assert_allclose(psess.z, z) + + +def _native_session(xy: np.ndarray, z: np.ndarray): + from track2data.readers.assemble import assemble_native_session + + return assemble_native_session( + session_id="pair", + folder=Path("pair.csv"), + reader="anipose", + fps=25.0, + raw_xy=xy, + raw_z=z, + source_unit="cm", + target_unit="cm", + vertical_axis="z", + flip=False, + keypoints=None, + tracking_intervals=None, + trajectory_source=None, + trajectory_format="anipose_pose3d_csv", + ) def test_t2d_add_names_the_unreadable_native_file(tmp_path: Path) -> None: diff --git a/tests/test_readers/native3d_files.py b/tests/test_readers/native3d_files.py index 3dae0ee..5680e24 100644 --- a/tests/test_readers/native3d_files.py +++ b/tests/test_readers/native3d_files.py @@ -17,22 +17,35 @@ def write_anipose_csv( score: np.ndarray | None = None, extra_columns: bool = True, frame_column: bool = True, + animals: Sequence[str] | None = None, ) -> Path: """Write an Anipose pose-3d CSV and return its path. *frames* is the list of ``fnum`` values (or a count for 0..n-1); *xyz* ``(F, K, 3)`` defaults to distinct, easy-to-read values (``100 * k + 10 * axis + frame``) and *score* ``(F, K)`` to 1. NaN in *xyz* is written as an empty cell, as Anipose does. + + With *animals* (prefixes) every column is ``__...`` and *xyz* is + ``(F, A, K, 3)`` / *score* ``(F, A, K)``; the default adds ``1000 * animal`` to the values. """ fnum = list(range(frames)) if isinstance(frames, int) else list(frames) n, k = len(fnum), len(keypoints) + prefixes = list(animals) if animals else [None] + a_count = len(prefixes) if xyz is None: base = np.arange(n, dtype=float)[:, None, None] - xyz = base + 100.0 * np.arange(k)[None, :, None] + 10.0 * np.arange(3)[None, None, :] + one = base + 100.0 * np.arange(k)[None, :, None] + 10.0 * np.arange(3)[None, None, :] + xyz = np.stack([one + 1000.0 * a for a in range(a_count)], axis=1) + elif animals is None: + xyz = xyz[:, None] if score is None: - score = np.ones((n, k)) + score = np.ones((n, a_count, k)) + elif animals is None: + score = score[:, None] + columns = [(a, j, (name if p is None else f"{p}_{name}")) for a, p in enumerate(prefixes) + for j, name in enumerate(keypoints)] header: list[str] = [] - for name in keypoints: + for _, _, name in columns: header += [f"{name}_x", f"{name}_y", f"{name}_z", f"{name}_error", f"{name}_ncams"] header += [f"{name}_score"] if extra_columns: @@ -47,9 +60,9 @@ def cell(value: float) -> str: lines = [",".join(header)] for row in range(n): cells: list[str] = [] - for j in range(k): - cells += [cell(xyz[row, j, a]) for a in range(3)] - cells += ["0.1", "4", cell(score[row, j])] + for a, j, _ in columns: + cells += [cell(xyz[row, a, j, ax]) for ax in range(3)] + cells += ["0.1", "4", cell(score[row, a, j])] if extra_columns: cells += ["1" if i == j else "0" for i in range(3) for j in range(3)] cells += ["0", "0", "0"] diff --git a/tests/test_readers/test_anipose.py b/tests/test_readers/test_anipose.py index 4feac98..260c917 100644 --- a/tests/test_readers/test_anipose.py +++ b/tests/test_readers/test_anipose.py @@ -281,3 +281,98 @@ def test_discover_warns_about_a_duplicate_column(tmp_path: Path) -> None: write_anipose_csv(other / "c.csv", KPS, 3) (group,) = scan([other]).groups assert group.detections[0].sessions[0].warnings == () + + +# โ”€โ”€ several animals in one file: a column prefix per animal (D-053) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ + + +def _two_fish(tmp_path: Path, **kwargs) -> Path: + return write_anipose_csv( + tmp_path / "pair.csv", KPS, 6, animals=["fishA", "fishB"], **kwargs + ) + + +def test_two_animals_in_one_file_become_two_slots_in_one_session(tmp_path: Path) -> None: + s = read(_two_fish(tmp_path), animals="fishA,fishB", keypoint="l-base", target_unit="mm") + assert s.n_animals == 2 and s.raw_xy.shape == (6, 2, 2) and s.raw_z.shape == (6, 2) + # default values: frame + 100 k + 10 axis + 1000 animal; the file's axes are x, y, z + for a in range(2): + np.testing.assert_allclose(s.raw_xy[:, a, 0], np.arange(6) + 1000.0 * a) + np.testing.assert_allclose(s.raw_xy[:, a, 1], np.arange(6) + 10.0 + 1000.0 * a) + np.testing.assert_allclose(s.raw_z[:, a], np.arange(6) + 20.0 + 1000.0 * a) + assert s.has_stable_identities is True + assert s.identities_labels == ["fishA", "fishB"] + assert s.keypoints is not None and s.keypoints.xy.shape == (6, 2, 3, 3) + assert s.keypoints.names == KPS # the animal prefix is not part of the keypoint name + + +def test_the_order_of_the_option_is_the_order_of_the_slots(tmp_path: Path) -> None: + s = read(_two_fish(tmp_path), animals="fishB, fishA", keypoint="l-base", target_unit="mm") + assert s.identities_labels == ["fishB", "fishA"] + np.testing.assert_allclose(s.raw_xy[:, 0, 0], np.arange(6) + 1000.0) + + +def test_prefixes_may_contain_underscores_and_the_longest_wins(tmp_path: Path) -> None: + path = write_anipose_csv( + tmp_path / "u.csv", ["nose"], 4, animals=["fish", "fish_2"] + ) + s = read(path, animals="fish,fish_2", keypoint="nose", target_unit="mm") + assert s.n_animals == 2 + np.testing.assert_allclose(s.raw_xy[:, 1, 0], np.arange(4) + 1000.0) + + +def test_a_missing_animal_is_refused_with_the_columns_it_lacks(tmp_path: Path) -> None: + with pytest.raises(Track2DataError) as err: + read(_two_fish(tmp_path), animals="fishA,fishC") + assert err.value.code == "ANIPOSE_ANIMAL_COLUMNS" and "fishC" in str(err.value) + + +def test_animals_must_share_their_keypoints(tmp_path: Path) -> None: + path = _two_fish(tmp_path) + text = path.read_text().replace("fishB_tail_tip_", "fishB_tail_end_") + path.write_text(text) + with pytest.raises(Track2DataError) as err: + read(path, animals="fishA,fishB") + assert err.value.code == "ANIPOSE_ANIMAL_COLUMNS" + + +def test_a_column_that_belongs_to_no_listed_animal_is_refused(tmp_path: Path) -> None: + with pytest.raises(Track2DataError) as err: + read(_two_fish(tmp_path), animals="fishA") # fishB's columns are left over + assert err.value.code == "ANIPOSE_ANIMAL_COLUMNS" and "fishB" in str(err.value) + + +def test_duplicate_animal_names_are_refused(tmp_path: Path) -> None: + with pytest.raises(Track2DataError) as err: + read(_two_fish(tmp_path), animals="fishA,fishA") + assert err.value.code == "READER_OPTION_INVALID" + + +def test_a_single_animal_file_still_reads_exactly_as_before(tmp_path: Path) -> None: + path = write_anipose_csv(tmp_path / "one.csv", KPS, 5) + plain = read(path, keypoint="l-base", target_unit="mm") + assert plain.n_animals == 1 and plain.identities_labels is None + # an animals list of one prefix-free name is not a thing: the option left out is the default + assert read(path, animals="", keypoint="l-base", target_unit="mm").raw_xy.shape == (5, 1, 2) + + +def test_a_missing_keypoint_for_one_animal_is_nan_for_that_animal_only(tmp_path: Path) -> None: + xyz = np.zeros((6, 2, 3, 3)) + np.arange(6)[:, None, None, None] + xyz[2:4, 1, 0, :] = np.nan + path = write_anipose_csv(tmp_path / "gap.csv", KPS, 6, animals=["a", "b"], xyz=xyz) + s = read(path, animals="a,b", keypoint="l-base", target_unit="mm") + assert np.isnan(s.raw_xy[2:4, 1]).all() and np.isfinite(s.raw_xy[2:4, 0]).all() + + +def test_a_score_cutoff_applies_per_animal(tmp_path: Path) -> None: + score = np.ones((6, 2, 3)) + score[1, 1, :] = 0.1 + path = write_anipose_csv(tmp_path / "s.csv", KPS, 6, animals=["a", "b"], score=score) + s = read(path, animals="a,b", keypoint="l-base", likelihood_cutoff=0.5, target_unit="mm") + assert np.isnan(s.raw_xy[1, 1]).all() and np.isfinite(s.raw_xy[1, 0]).all() + + +def test_discover_lists_the_animals_option_beside_the_others(tmp_path: Path) -> None: + _two_fish(tmp_path) + (group,) = scan([tmp_path]).groups + assert "animals" in [p.name for p in group.best.parameters] diff --git a/track2data/api.py b/track2data/api.py index 1971d96..5de8906 100644 --- a/track2data/api.py +++ b/track2data/api.py @@ -1365,7 +1365,11 @@ def view_skipped_metrics( depth_height_cm=psess.depth_height_cm, ) if psess.z is not None: - scale.update(dimension="3d", native_3d=True) + scale.update( + dimension="3d", + native_3d=True, + native_several_animals=psess.xy.shape[1] >= 2, + ) return view_skipped_metrics( selected, self.camera_view_for_psess(psess), diff --git a/track2data/core/models.py b/track2data/core/models.py index 5e0c1f2..8ffb3cf 100644 --- a/track2data/core/models.py +++ b/track2data/core/models.py @@ -791,6 +791,10 @@ class PPStepResult: affected_frames: int affected_per_individual: list[int] notes: str = "" + #: Identity-switch correction only: ``(n_frames, n_animals)`` ints, the input column that holds + #: each identity at each frame, so arrays that follow the positions (a native 3-D z) can be + #: relabelled the same way. None for every other step. + permutation: Any = None @dataclass @@ -946,8 +950,8 @@ class PreprocessedSession: # session's coordinate unit, laid out on the same rows as ``xy`` (NaN on inserted rows). # None for every other session. It is the file's coordinate cleaned like ``xy``: blanked # where a jump was replaced, then bridged, gap-filled and smoothed with the same settings - # (preprocess/z_axis.py). Identity-switch correction never touches it, so it is only valid - # for single-animal sources (native files are one animal). + # (preprocess/z_axis.py). Identity-switch correction relabels it with x and y (the pipeline + # applies the step's permutation), so it stays with its animal in a file with several. z: np.ndarray | None = None # (n_frames, n_animals) vertical speed in the project length unit per second, from # ``compute_kinematics`` on the WHOLE session's Z with the manifest's KinematicsCfg diff --git a/track2data/metrics/availability.py b/track2data/metrics/availability.py index 9634ebd..e836edd 100644 --- a/track2data/metrics/availability.py +++ b/track2data/metrics/availability.py @@ -125,6 +125,7 @@ def depth_scale_reason( px_per_cm: float | None, depth_height_cm: float | None, native_3d: bool = False, + native_several_animals: bool = False, ) -> str | None: """Why a ``requires_depth_scale`` metric cannot run, or None (also for any other metric). @@ -132,7 +133,8 @@ def depth_scale_reason( cm scale (``bodylength``), then a unit without ``px_per_cm`` / ``depth_height_cm``. *native_3d* marks native 3-D data (its ``z`` is absolute, in the project unit), which needs - no ``depth_height_cm``. + no ``depth_height_cm``. *native_several_animals* is read by :func:`unavailable_reason` (group + metrics) and has no effect here. """ if not getattr(metric_cls, "requires_depth_scale", False): return None @@ -172,6 +174,10 @@ def manifest_depth_scale(manifest: Any) -> dict[str, Any]: } if native: scale["native_3d"] = True + from track2data.readers.native3d import manifest_native_several_animals + + if manifest_native_several_animals(manifest): + scale["native_several_animals"] = True return scale @@ -193,7 +199,9 @@ def unavailable_reason( if depth_scale is not None and depth_scale.get("native_3d", False): if getattr(metric_cls, "id", None) in PIXEL_FRAME_METRIC_IDS: return NEEDS_PIXEL_FRAME_REASON - if getattr(metric_cls, "level", None) == "group": + if getattr(metric_cls, "level", None) == "group" and not depth_scale.get( + "native_several_animals", False + ): return NEEDS_SEVERAL_ANIMALS_REASON if getattr(metric_cls, "uses_depth", False) and not has_depth: return NEEDS_WATER_COLUMN_REASON diff --git a/track2data/preprocess/identity_switch.py b/track2data/preprocess/identity_switch.py index d5d1838..049f3b5 100644 --- a/track2data/preprocess/identity_switch.py +++ b/track2data/preprocess/identity_switch.py @@ -105,7 +105,6 @@ def correct_switches( xy: np.ndarray, cfg: IdSwitchCfg, swap_boundaries: set[int] | None = None, - has_z: bool = False, ) -> tuple[np.ndarray, PPStepResult]: """Correct identity switches in multi-animal trajectory data. @@ -142,17 +141,14 @@ def correct_switches( the permutation it licenses applies from *b* onward. When ``None``, every frame is a candidate -- correct, but far more false-positive-prone, so prefer supplying it. - has_z: - The session also carries a vertical coordinate (native 3-D). It is never - re-labelled here, so with several animals it can end up on the wrong - identity; that is logged once per call. - Returns ------- out: New array (input not mutated) with corrected identities. result: - ``PPStepResult`` describing corrected frames. + ``PPStepResult`` describing corrected frames. Its ``permutation`` says which input + column holds each identity at each frame, so an array that belongs to the positions (a + native 3-D session's z) can be relabelled the same way. """ n_frames, n_animals, _ = xy.shape @@ -172,14 +168,6 @@ def correct_switches( else "geometry alone, with no fragment boundaries to constrain it", ) - if has_z: - logger.warning( - "Identity-switch correction re-labels x and y only; the 3-D z coordinate of this " - "session (%d animals) is left as read, so it can end up on the wrong identity. " - "Identity-switch correction is meant for a single animal's z, or sessions without z.", - n_animals, - ) - identity = np.arange(n_animals) # ``active[k]`` is the column of the *input* array that currently holds @@ -253,12 +241,14 @@ def correct_switches( # provenance). affected_frames = 0 affected_per_individual = [0] * n_animals + columns = np.tile(identity, (n_frames, 1)) for i, (start, perm) in enumerate(segments): end = segments[i + 1][0] if i + 1 < len(segments) else n_frames if np.array_equal(perm, identity): continue out[start:end] = xy[start:end][:, perm, :] + columns[start:end] = perm span = end - start affected_frames += span for k in range(n_animals): @@ -269,4 +259,5 @@ def correct_switches( step_name="identity_switch", affected_frames=affected_frames, affected_per_individual=affected_per_individual, + permutation=columns, ) diff --git a/track2data/preprocess/pipeline.py b/track2data/preprocess/pipeline.py index baf2a08..c0c696a 100644 --- a/track2data/preprocess/pipeline.py +++ b/track2data/preprocess/pipeline.py @@ -162,9 +162,11 @@ def _checkpoint(_step: str) -> None: rows = np.flatnonzero(np.isin(expansion.frame_index, list(swap_boundaries))) swap_boundaries = {int(r) for r in rows} xy, step = correct_switches( - xy, config.identity_switch, swap_boundaries=swap_boundaries, - has_z=session.raw_z is not None, + xy, config.identity_switch, swap_boundaries=swap_boundaries ) + if z_rows is not None and step.permutation is not None: + # the vertical coordinate travels with its animal's x and y + z_rows = np.take_along_axis(z_rows, step.permutation, axis=1) report.steps.append(step) # 4. Smoothing diff --git a/track2data/readers/anipose.py b/track2data/readers/anipose.py index 2075fd0..9745d63 100644 --- a/track2data/readers/anipose.py +++ b/track2data/readers/anipose.py @@ -8,8 +8,11 @@ What the file does **not** record is the frame rate and the unit of its coordinates (the unit of the calibration board the user used; mm is the usual one, but it is a fact about the recording -rather than the file). Both are options the user gives, never defaulted. There is one animal per -file; several animals are several files. +rather than the file). Both are options the user gives, never defaulted. A file holds one animal +unless the user lists the animals' column prefixes in the ``animals`` option (decision D-053): +``fishA,fishB`` reads the columns ``fishA__x|y|z`` and ``fishB__x|y|z`` as two +slots of one session, in that order. The prefixes are never guessed from the names, because +``left_ear`` / ``right_ear`` look the same as ``fishA_nose`` / ``fishB_nose``. Rows missing from ``fnum`` are not filled: the stored rows are the frames the file has, and the gaps become ``tracking_intervals``, so no step treats the move across a gap as one frame of motion. @@ -126,7 +129,18 @@ class AniposeReader(SessionReader): verification = "synthetic_only" coordinate_frame = "physical_unaligned" provides_native_3d: ClassVar[bool] = True - parameters: ClassVar[tuple[ReaderParameter, ...]] = native_parameters("mm") + parameters: ClassVar[tuple[ReaderParameter, ...]] = ( + *native_parameters("mm"), + ReaderParameter( + name="animals", + label="Animal column prefixes (several animals in one file)", + kind="str", + help="Leave empty for a file with one animal. For several animals in one file, " + "list the prefixes of their columns in order, separated by commas: 'fishA,fishB' " + "reads fishA__x, _y, _z and fishB__x, _y, _z. Every animal " + "needs the same keypoints.", + ), + ) # โ”€โ”€ detection โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ @@ -270,6 +284,77 @@ def _read_table(path: Path, layout: _Layout) -> pd.DataFrame: return numeric +def animal_prefixes(value: Any) -> list[str]: + """The prefixes in the ``animals`` option, in order; empty when the option is not used.""" + if value is None: + return [] + names = [part.strip() for part in str(value).split(",") if part.strip()] + if len(set(names)) != len(names): + raise ImportError_( + f"Option 'animals' repeats a name: {value!r}.", + code="READER_OPTION_INVALID", + subject="animals", + remediation="List each animal prefix once, e.g. fishA,fishB.", + ) + return names + + +def _split_animals( + path: Path, layout: _Layout, prefixes: list[str] +) -> tuple[list[str], list[list[str]]]: + """Per animal, the file's column stem of each shared keypoint, and the shared names. + + A keypoint column belongs to the longest listed prefix it starts with (``fish_`` and + ``fish_2_`` can both be animals). Every column must belong to one, and every animal must have + the same keypoints; otherwise the file is not what the user said it is. + """ + by_animal: list[list[str]] = [[] for _ in prefixes] + stray: list[str] = [] + for name in layout.keypoints: + owners = [ + (len(p), i) for i, p in enumerate(prefixes) if name.startswith(f"{p}_") and name != p + ] + if not owners: + stray.append(name) + continue + by_animal[max(owners)[1]].append(name) + problems: list[str] = [] + for prefix, stems in zip(prefixes, by_animal, strict=True): + if not stems: + problems.append(f"no columns start with '{prefix}_'") + if stray: + problems.append( + "columns of no listed animal: " + + ", ".join(stray[:4]) + + (" ..." if len(stray) > 4 else "") + ) + shared: list[str] = [] + if not problems: + shortened = [ + [s[len(p) + 1 :] for s in stems] + for p, stems in zip(prefixes, by_animal, strict=True) + ] + shared = shortened[0] + for p, names in zip(prefixes[1:], shortened[1:], strict=True): + if names != shared: + missing = sorted(set(shared) ^ set(names)) + problems.append( + f"'{p}' does not have the same keypoints as '{prefixes[0]}' " + f"({', '.join(missing[:4])})" + ) + if problems: + raise _error( + "ANIPOSE_ANIMAL_COLUMNS", + f"{path.name}: the animals option does not match the columns: " + + "; ".join(problems) + + ".", + "List every animal's column prefix in the order you want the animals, and give each " + "the same keypoints; or leave the option empty for a file with one animal.", + str(path), + ) + return shared, by_animal + + def _build_session(given: Path, path: Path, layout: _Layout, opts: dict[str, Any]) -> Session: """The session for one file. ``raw_xy``/``raw_z`` follow ``vertical_axis`` and ``flip``; the stored skeleton keeps the file's axes (converted to the target unit, not permuted or flipped). @@ -297,21 +382,37 @@ def _build_session(given: Path, path: Path, layout: _Layout, opts: dict[str, Any ) order = np.argsort(fnum, kind="stable") frames = fnum[order].astype(np.int64) - names = list(layout.keypoints) + prefixes = animal_prefixes(opts.get("animals")) + if prefixes: + names, stems_by_animal = _split_animals(path, layout, prefixes) + else: + names, stems_by_animal = list(layout.keypoints), [list(layout.keypoints)] xyz = np.stack( [ - table[[f"{n}_x", f"{n}_y", f"{n}_z"]].to_numpy(dtype=np.float64)[order] - for n in names + np.stack( + [ + table[[f"{n}_x", f"{n}_y", f"{n}_z"]].to_numpy(dtype=np.float64)[order] + for n in stems + ], + axis=1, + ) + for stems in stems_by_animal ], axis=1, - ) # (R, K, 3) + ) # (R, A, K, 3) xyz[~np.isfinite(xyz).all(axis=-1)] = np.nan # a triple is missing as a whole - xyz = convert_units(xyz, opts["unit"], opts["target_unit"])[:, None, :, :] # (R, 1, K, 3) + xyz = convert_units(xyz, opts["unit"], opts["target_unit"]) score = None if layout.scored: score = np.stack( - [table[f"{n}_score"].to_numpy(dtype=np.float64)[order] for n in names], axis=1 - )[:, None, :] # (R, 1, K) + [ + np.stack( + [table[f"{n}_score"].to_numpy(dtype=np.float64)[order] for n in stems], axis=1 + ) + for stems in stems_by_animal + ], + axis=1, + ) # (R, A, K) cutoff = opts.get("likelihood_cutoff") if cutoff and score is None: raise DataValidationError( @@ -347,6 +448,8 @@ def _build_session(given: Path, path: Path, layout: _Layout, opts: dict[str, Any trajectory_source=path, trajectory_format="anipose_pose3d_csv", ) + if prefixes: + session.identities_labels = list(prefixes) if opts.get("keep_skeleton") and keypoints is None and session.raw_attrs is not None: session.raw_attrs["skeleton_not_kept"] = ( "the full skeleton is larger than the storage budget; only the chosen keypoint was kept" diff --git a/track2data/readers/native3d.py b/track2data/readers/native3d.py index cda468f..16bb994 100644 --- a/track2data/readers/native3d.py +++ b/track2data/readers/native3d.py @@ -54,6 +54,22 @@ def manifest_native_3d(manifest: Any) -> bool: return any(ref.reader in NATIVE_3D_READERS for ref in manifest.sessions) +def ref_animal_count(ref: Any) -> int: + """How many animals a native entry's file holds, from its saved ``animals`` option (1 when + it is empty or the entry is of another reader): the number of comma-separated prefixes.""" + if ref.reader not in NATIVE_3D_READERS: + return 1 + names = [p.strip() for p in str(ref.reader_options.get("animals") or "").split(",")] + return max(1, len({n for n in names if n})) + + +def manifest_native_several_animals(manifest: Any) -> bool: + """Whether every native session of *manifest* holds several animals, so group metrics have + animals to compare. One single-animal native file rules them out for the project.""" + refs = [r for r in manifest.sessions if r.reader in NATIVE_3D_READERS] + return bool(refs) and all(ref_animal_count(r) >= 2 for r in refs) + + def ref_vertical_axis(ref: Any) -> str | None: """The vertical axis a native entry's file is read with, from its saved options (None for an entry of another reader). Confirmed entries save only the options that were given, so a From a1f80686bfbb9ce7919efd7f6d2fb27bc8c09d00 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 16 Oct 2026 07:08:26 +0000 Subject: [PATCH 2/2] fix(core): hash a project the same on every platform The project hash dumped folder paths with the platform separator, so the two golden-hash tests failed on Windows on every push to main. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01N7hYfMaQFqSMBtq9ZWJ3gV --- CHANGELOG.md | 1 + track2data/core/models.py | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a4866d..821f4d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ### Fixed +- **A project hashes the same on Windows and elsewhere.** The project hash wrote folder paths with the platform's separators, so a project folder hashed differently on Windows (and two golden-hash tests failed there on every push to `main`). It now writes them with forward slashes. Hashes on Linux and macOS are unchanged; on Windows a project is re-hashed once, which refreshes its preprocessing cache. - **idtracker.ai v5 sessions with no recorded frame rate can be given one.** The legacy v5 reader now takes an `fps` option (`--option fps=25`, or the Frame rate field on import), used only after `video_object.npy` and the session json have been tried. Without any of the three it still stops with `VIDEO_FPS_MISSING` rather than assuming 1 fps; the message now says how to supply it. - **Rescanning a folder that was split into panels** offers the whole-frame session again instead of reporting it "already in the project". The import dialog and the project store now share one duplicate rule (decision D-046); a true duplicate is still left out. - **idtracker.ai pickle reader could run code.** The allowlist for `list_of_blobs.pickle` admitted every class in any module whose name starts with `numpy`, including helpers that execute code. It is now an exact list of the array-reconstruction functions numpy pickles use. diff --git a/track2data/core/models.py b/track2data/core/models.py index 8ffb3cf..1e3ea25 100644 --- a/track2data/core/models.py +++ b/track2data/core/models.py @@ -771,8 +771,10 @@ class ProjectManifest(BaseModel): def project_hash(self) -> str: """16-char hex hash of the manifest content (timestamps excluded).""" + # JSON mode writes a path with forward slashes on every platform, so the same project + # hashes the same on Windows and elsewhere. data: dict[str, Any] = self.model_dump( - exclude={"created_at", "updated_at", "run_log_path"} + mode="json", exclude={"created_at", "updated_at", "run_log_path"} ) serialised = json.dumps(data, sort_keys=True, default=str) return hashlib.sha256(serialised.encode()).hexdigest()[:16]