Several animals in one Anipose file

CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ### Added
 
+- **Several animals in one Anipose file.** List the animals' column prefixes in the new **Animal column prefixes** option (`--option animals=fishA,fishB`) and the file's `fishA_<keypoint>_x|y|z` and `fishB_...` columns become two animals of one session, in that order. Group metrics are then offered for the project when every native file holds several animals. Identity-switch correction now relabels a native file's vertical coordinate together with x and y. Files without the option read exactly as before (decision D-053).
 - **Three more 3-D metrics.** IL-18 *Vertical Speed* (mean, median and maximum of |dz/dt| in cm/s, plus body lengths per second), GL-18 *3-D Inter-Individual Distance* (mean over all pairs, the 3-D counterpart of GL-2) and GL-19 *3-D Group Cohesion* (reciprocal of the 3-D nearest-neighbour or inter-individual distance, chosen by `cohesion_source`). They need a fused or native 3-D session and a depth scale, skip frames with a missing animal as GL-16 does, and are greyed out otherwise. The registry now holds 57 metrics (decision D-052).
 - **Warning before a run when most animal slots are empty.** A multi-animal session in which at least half the (frame, animal) slots have no position, typically a Ctrax file with one slot per track fragment, now raises an advisory in `track2data validate`, the log and the run README, saying that group metrics skip any frame with a missing animal and pointing to the `top_n` option.
 - **Metadata: Excel workbooks with several sheets.** The metadata screen loads `.xlsx` files (it only took CSV before) and, when the workbook has more than one sheet, shows a **Sheet** menu; the choice is saved with the project. Projects without a chosen sheet are unchanged, including their project hash. The old `.xls` format is refused with a message that says to save as `.xlsx` or CSV.
@@ -24,6 +25,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ### Fixed
 
+- **A project hashes the same on Windows and elsewhere.** The project hash wrote folder paths with the platform's separators, so a project folder hashed differently on Windows (and two golden-hash tests failed there on every push to `main`). It now writes them with forward slashes. Hashes on Linux and macOS are unchanged; on Windows a project is re-hashed once, which refreshes its preprocessing cache.
 - **idtracker.ai v5 sessions with no recorded frame rate can be given one.** The legacy v5 reader now takes an `fps` option (`--option fps=25`, or the Frame rate field on import), used only after `video_object.npy` and the session json have been tried. Without any of the three it still stops with `VIDEO_FPS_MISSING` rather than assuming 1 fps; the message now says how to supply it.
 - **Rescanning a folder that was split into panels** offers the whole-frame session again instead of reporting it "already in the project". The import dialog and the project store now share one duplicate rule (decision D-046); a true duplicate is still left out.
 - **idtracker.ai pickle reader could run code.** The allowlist for `list_of_blobs.pickle` admitted every class in any module whose name starts with `numpy`, including helpers that execute code. It is now an exact list of the array-reconstruction functions numpy pickles use.

README.md

@@ -34,7 +34,7 @@ It is a desktop application and a Python engine you can drive headlessly for bat
 **Reads** idtracker.ai (6.x and v5), DeepLabCut (CSV, also Lightning Pose), SLEAP
 (analysis HDF5), Ctrax (raw `.mat`), `trx.mat`, and two native 3-D formats, Anipose
 `pose-3d` CSV and DeepLabCut 3-D tables (positions in physical units; the 3-D metrics
-the 3-D metrics IL-16 and IL-17 run on them, and IL-15 once you give the water column; GL-16 does not, because a native file holds one animal). Run
+the 3-D metrics IL-16 and IL-17 run on them, and IL-15 once you give the water column; GL-16 does not, because a native file holds one animal unless you list the animals of an Anipose file). Run
 `track2data list-readers` for the list and the options each reader needs.
 
 **Individual** — distance travelled, speed, acceleration, tortuosity,

docs/3d-movement/2026-10-10-native-3d-files-design.md

@@ -23,8 +23,11 @@ A adds readers for both and lets the 3-D metrics of B2 and IL-15 run on them.
   `anipose` (pose-3d CSV) and `deeplabcut_3d`. There are no real sample files at hand for the DLC-3D
   layout. For Anipose the repo's published sample (`tests/real_samples/`) is used by the existing
   reader-contract case when the fixture is present; it is not required.
-- **One animal per file.** Neither format carries an animal axis. Several animals are several files
-  (several sessions), as for the other single-animal readers.
+- **One animal per file, unless the user lists the animals (Anipose, D-053).** Neither format carries
+  an animal axis. Several animals are several files (several sessions), as for the other
+  single-animal readers. An Anipose file can hold several animals as column prefixes
+  (`<animal>_<keypoint>_x|y|z`) when the `animals` option names them in order; the prefixes are
+  never guessed, since `left_ear` / `right_ear` look the same as `fishA_nose` / `fishB_nose`.
 - **Units are converted when reading, to the project's length unit.** The reader takes `unit` (the
   file's unit: mm, cm, m, in; Anipose proposes mm, which the file does not state, so it is always
   confirmed) and `target_unit` (default cm). The first import sets the project's
@@ -139,7 +142,7 @@ code does.
 - **IL-17 reads `z`.** The first build left IL-17 on `depth * height`, so a native session without the
   water column got NaN; found by the end-to-end test and fixed. IL-15 outside values, and non-finite z,
   are NaN and counted.
-- **GL-16 is not offered for native files** (one animal per file; the metric needs two in a session).
+- **GL-16 is not offered for native files** unless every native file of the project lists several animals (the metric needs two in a session).
 - **The unreadable-format notice for DLC-3D was narrowed** in `readers/recognise.py`: the single-animal
   tables the reader takes are no longer reported, while a multi-animal table (extra `individuals` header
   row) or one without full x, y, z triples is still named as one that cannot be read yet.

docs/INTEROPERABILITY.md

@@ -110,7 +110,7 @@ rides along as an attribute.
 | Ctrax raw `.mat` input | ✅ supported | The file Ctrax's *Save Tracks as Matlab File* writes. Frame size is an option you give; the frame rate comes from the file. Ctrax track ids are fragments, so a session is identity-free and one slot per id (`top_n` keeps the longest). See `docs/tracker-formats/README.md`. |
 | `trx.mat` input (Ctrax / FlyTracker / JAABA) | 🧪 synthetic only | A classic (v5/v7) MATLAB file holding a `trx` struct, built from the documented layout with no real sample pinned yet. Frame size is an option; `fps` is read from the file. `pxpermm` (pixels per mm) becomes the session scale for Session calibration, pending the usual unit confirmation; a value of exactly 1 is ignored. v7.3 files are not read. |
 | DeepLabCut `.h5` input | 🧪 synthetic only | The `.h5` DeepLabCut writes beside the CSV (pandas table under `df_with_missing`). Read with h5py; the pickled column labels are decoded by an unpickler that refuses every global, and PyTables is not needed (D-050). The frame rate and frame size are options you give; the keypoint is typed, not picked from a list. A CSV of the same name beside it is the one session. Multi-animal tracklet files (`*_el.h5`) are refused. Becomes *supported* once the real EPM sample has been read in a run. |
-| Anipose pose-3d CSV input | 🧪 synthetic only | One animal per file, native 3-D (X, Y, Z in the file's unit). Built from the documented layout; no real sample pinned yet. Several animals in one file are not read. |
+| Anipose pose-3d CSV input | 🧪 synthetic only | Native 3-D (X, Y, Z in the file's unit). One animal per file, or several when the `animals` option lists their column prefixes (`<animal>_<keypoint>_x|y|z`). Built from the documented layout; no real sample pinned yet. |
 | DeepLabCut-3D input | 🧪 synthetic only | The triangulated 3-D table DeepLabCut writes, read as native 3-D. Built from the documented layout; no real sample pinned yet. |
 | NWB | ❌ not planned | Would be worth revisiting if the behavioural-NWB extensions stabilise. |
 

docs/METRICS_SPEC.md

@@ -631,7 +631,7 @@ info-button modal (§6).
 
 ### 4.3 Social spacing
 
-**Availability (native 3-D projects).** Every group metric (`GL-*`) is unavailable in a native 3-D project (Anipose, DeepLabCut 3-D files): "needs several animals in one session; a native 3-D file holds one animal". The Metrics screen greys them out with that reason, and `validate` and the run `README.md` list it.
+**Availability (native 3-D projects).** Every group metric (`GL-*`) is unavailable in a native 3-D project (Anipose, DeepLabCut 3-D files) unless every file holds several animals (an Anipose file with its `animals` option set); otherwise: "needs several animals in one session; a native 3-D file holds one animal". The Metrics screen greys them out with that reason, and `validate` and the run `README.md` list it.
 
 #### GL-1 — Nearest-neighbour distance (NND)
 

docs/dev/DECISIONS.md

@@ -1256,3 +1256,11 @@ could not be re-added, while the store would have accepted it.
 
 **Why:** one frame rule and one body-length rule keep the three group metrics comparable, and the NND default keeps GL-19 the direct 3-D analogue of GL-6.
 
+### D-053 · Several animals in one Anipose file: a column prefix per animal, named by the user
+
+**Status:** accepted (the user chose option (a)); implemented. Synthetic files only, like the rest of the reader.
+
+**Decision:** the Anipose reader takes an `animals` option, a comma-separated list of column prefixes (`fishA,fishB`). A keypoint column belongs to the longest listed prefix it starts with (`<prefix>_<keypoint>_x|y|z`); the list order is the slot order and the labels are stored as `identities_labels`. Every animal must have the same keypoints and every keypoint column must belong to a listed animal, else `ANIPOSE_ANIMAL_COLUMNS`. The option left empty reads one animal, exactly as before. Group metrics are offered for a native project when every native session lists two or more animals (`manifest_native_several_animals`); for a run, the preprocessed session's animal count decides. Identity-switch correction reports the permutation it applied (`PPStepResult.permutation`) and the pipeline applies it to the native z, so z stays with its animal.
+
+**Why:** the prefixes are not guessed because a single-animal file with `left_ear` and `right_ear` has the same shape as two animals with one keypoint each; guessing would turn a one-animal file into two. A project with one single-animal native file keeps the group metrics off, since a group metric on a one-animal session has nothing to compare.
+

docs/guide/USER_GUIDE.md

@@ -788,14 +788,14 @@ no fusion): the positions are `(x, y, z)` in the project's unit, and IL-16 and I
 column. Example: a fish that moves (3, 4, 12) mm per frame in an Anipose file at 25 frames per second,
 read as cm, steps hypot(0.3, 0.4, 1.2) = 1.3 cm per frame, so 10 steps give an IL-16 of 13 cm and an
 IL-17 of 1.3 x 25 = 32.5 cm/s; its 2-D metrics measure the horizontal steps of 0.5 cm (IL-1 5 cm, IL-2
-12.5 cm/s). **GL-16 is not offered for native files** (it needs two animals in one session and a file holds one), and neither is any other group metric ("needs several animals in one session; a native 3-D file holds one animal"). **IL-3, IL-9, IL-10 and IL-14 are not offered either** ("uses a pixel-sized default and the project's native 3-D files have no pixel frame"): their default thresholds are in pixels and would silently change with the project unit. The Metrics screen greys them out, `validate` and the run `README.md` list the reason. IL-15 appears once the water column is set.
+12.5 cm/s). **GL-16 is not offered for native files that hold one animal** (it needs two animals in one session), and neither is any other group metric ("needs several animals in one session; a native 3-D file holds one animal"). An Anipose file that holds several animals, with columns named `<animal>_<keypoint>_x`, `_y`, `_z`, offers them when you list the prefixes in the **Animal column prefixes** option (`fishA,fishB`, in the order you want the animals); every native file in the project must then do the same. **IL-3, IL-9, IL-10 and IL-14 are not offered either** ("uses a pixel-sized default and the project's native 3-D files have no pixel frame"): their default thresholds are in pixels and would silently change with the project unit. The Metrics screen greys them out, `validate` and the run `README.md` list the reason. IL-15 appears once the water column is set.
 The per-frame table has a `z_cm` column, the file's own vertical coordinate after the same gap filling
 and smoothing as x and y (so IL-16 can be reproduced from `x_cm`, `y_cm` and `z_cm`); with a water
 column it also carries `depth_fraction` and `depth_cm` as for a fused unit. The run's `README.md` has a
 **Native 3-D source** section (reader, file unit, project unit, vertical axis, flip, water column or its
 absence) and, with a water column, a row **Samples outside the water column** with the count left out.
 
-Limits of native 3-D files: one animal per file; no mixing with pixel sessions; the unit you give
+Limits of native 3-D files: one animal per file (several only for an Anipose file whose animal prefixes you list); no mixing with pixel sessions; the unit you give
 must be the one of the calibration board (a wrong one scales everything); no refraction correction;
 the water column is entered, not estimated; no zones or background image; no group metrics and no IL-3, IL-9, IL-10 or IL-14; a multi-animal DeepLabCut
 3-D table (with an `individuals` header row) is named by the scan as one it cannot read yet.

docs/wiki/Compatibility.md

@@ -14,7 +14,7 @@ is authoritative.
 | **DeepLabCut** (CSV; also Lightning Pose) | **Supported** | One session per file. Frame rate and frame size are required options; the optional **Scale (pixels per cm)** feeds Session calibration |
 | **SLEAP** (analysis HDF5) | **Supported** | Export Analysis HDF5 from SLEAP. Same required options and optional scale as DeepLabCut |
 | **Ctrax** (raw `.mat`) | **Supported** | Frame size is a required option; the frame rate comes from the file. Ctrax track ids are fragments, so a session is identity-free. Optional scale as above |
-| **Anipose** (`pose-3d` CSV) | **Unverified** | Built from the published layout, tested on synthetic files only. Native 3-D: one animal per file; the frame rate and the unit of the coordinates are required options (never guessed). See the 3-D section of the user guide |
+| **Anipose** (`pose-3d` CSV) | **Unverified** | Built from the published layout, tested on synthetic files only. Native 3-D: one animal per file, or several when you list their column prefixes in the `animals` option; the frame rate and the unit of the coordinates are required options (never guessed). See the 3-D section of the user guide |
 | **DeepLabCut 3-D** (triangulated CSV) | **Unverified** | Same as Anipose; the unit is the calibration board's and is not proposed. Multi-animal 3-D tables are not read (the scan names them) |
 | **Ctrax / FlyTracker / JAABA** (`trx.mat`) | **Unverified** | Built from the documented layout with no real sample yet, and labelled as such in the confirm step. Classic MATLAB files only. `pxpermm` becomes the session scale |
 

tests/test_integration/test_native_3d.py

@@ -350,3 +350,38 @@ def test_il15_names_the_water_column_as_its_source(project: Path) -> None:
     psess = engine.preprocess(engine.import_ref(manifest.sessions[0]))
     result = engine.compute_metrics(psess)["IL-15"]
     assert set(result["depth_extent_source"]) == {"water_column"}
+
+
+# ── one file, several animals (prefix per animal) ───────────────────────────────────────────
+
+
+def test_two_animals_in_one_file_run_the_group_metrics(tmp_path: Path) -> None:
+    """Two fish 10 cm apart in x (100 mm), moving together: GL-16 sees them, IL-16 sees each."""
+    data = tmp_path / "data"
+    data.mkdir()
+    xyz = np.stack([_track(), _track() + np.array([100.0, 0.0, 0.0])], axis=1)  # (F, 2, 1, 3)
+    write_anipose_csv(
+        data / "pair.csv", ["head"], N_FRAMES, animals=["f1", "f2"], xyz=xyz
+    )
+    now = datetime.now(tz=UTC)
+    project = tmp_path / "p.t2d.json"
+    write_manifest(ProjectManifest(project_name="pair", created_at=now, updated_at=now), project)
+    result = CliRunner().invoke(
+        cli,
+        ["add", str(project), str(data), "--option", "unit=mm", "--option", "fps=25",
+         "--option", "animals=f1,f2", "--yes"],
+    )
+    assert result.exit_code == 0, result.output
+    manifest = read_manifest(project)
+    manifest = manifest.model_copy(
+        update={
+            "metrics": SELECTION.model_copy(update={"individual": ["IL-16"], "group": ["GL-16"]})
+        }
+    )
+    out = tmp_path / "out"
+    Engine(manifest).run(out, exporters=["csv_long"])
+    long = pd.read_csv(out / "pair" / "metrics_long.csv")
+    path = long[(long["metric_id"] == "IL-16") & (long["column"] == "path_length_3d_cm")]
+    assert sorted(path["individual_id"].tolist()) == [0, 1]
+    assert path["value"].tolist() == pytest.approx([PATH_CM, PATH_CM], abs=1e-9)
+    assert _value(long, "GL-16", "mean_nnd_3d_cm") == pytest.approx(10.0, abs=1e-9)

tests/test_native3d/test_gating.py

@@ -71,3 +71,27 @@ def test_pixel_and_fused_projects_still_offer_them() -> None:
     skipped = Engine(fused).skipped_metrics(False)
     reasons = {NEEDS_PIXEL_FRAME_REASON, NEEDS_SEVERAL_ANIMALS_REASON}
     assert not reasons & set(skipped.values())
+
+
+def _pair_ref(session_id: str):
+    ref = _ref(session_id)
+    return ref.model_copy(update={"reader_options": {**ref.reader_options, "animals": "a, b"}})
+
+
+def _group_reasons(refs) -> dict[str, str]:
+    manifest = _manifest(refs, calibration=SESSION_CM, individual=STILL_OFFERED, group=GROUP)
+    return {m: r for m, r in Engine(manifest).skipped_metrics(False).items() if m in GROUP}
+
+
+def test_group_metrics_are_offered_for_a_native_session_with_several_animals() -> None:
+    assert _group_reasons([_pair_ref("n1")]) == {}
+
+
+def test_group_metrics_stay_unavailable_for_a_one_animal_native_session() -> None:
+    reasons = _group_reasons([_ref("n1")])
+    assert set(reasons) == set(GROUP) and set(reasons.values()) == {NEEDS_SEVERAL_ANIMALS_REASON}
+
+
+def test_one_single_animal_file_rules_the_group_metrics_out_for_the_project() -> None:
+    reasons = _group_reasons([_pair_ref("n1"), _ref("n2")])
+    assert set(reasons) == set(GROUP)