Skip to content

Add PTT transitions, branch arbors, transitions I/O and a TRXViz bridge - #18

Merged
36000 merged 7 commits into
mainfrom
transitions-arbors-trxviz
Nov 17, 2026
Merged

36000 merged 7 commits into
mainfrom
transitions-arbors-trxviz

Conversation

@mattcieslak

Copy link
Copy Markdown
Collaborator

Summary

This branch adds the pieces two downstream users need from MPAT:

  • qsirecon gets the run options its MPAT pipeline exposes (PTT transitions, SIFT2-calibrated launches, branch arbors) and a way to save transition probabilities as intermediate derivatives (desc-transitions_dwimap.odx / .nii.gz).
  • TRXViz gets mpat.trxviz_bridge, the subprocess its MPAT nodes call to build transitions and solve a bundle, streaming the solve as it converges so the bundle can be watched forming.

qsirecon currently pins MPAT to af64577, which has none of these options. After this lands, its pin can move to this branch's merge commit.

What's in it

mpat run / CLI options (afq.py, cli.py)

  • --ptt switches to parallel-transport transition defaults (fod_thresh=0.08, fod_power=1.5, curv_sigma=18, probe_radius=0.5, stop_mass=1e-4). Each can also be set on its own with --step, --stop-mass, --fod-thresh, --fod-power, --curv-sigma and --probe-radius.
  • --sift2 calibrates seed launch weights with path-consistent SIFT2 (sift2_optimize_launches) before bundles are solved.
  • --arbor / --arbor-branches write a branch arbor per bundle (desc-arbor.trx, .json, .tsv).
  • --save-transitions {odx,nifti,both} persists the chain's transition probabilities.

Transitions I/O (transitions.py, chain.py)

  • save_transitions_odx / load_transitions_odx store a chain as an ODX archive: the 26-neighbour stencil, absorbing classes, seed launches, fixel lobes, whole-brain class densities, overlength survival and the fODF SH. A saved chain reloads without the original fODF NIfTI. The header records the chain config and the max length used for overlength states.
  • save_transitions_nifti / load_transitions_nifti write and read a 4-D (X, Y, Z, 26) transition NIfTI plus CoDI/CoAsy maps.
  • Chain.save_odx, Chain.load_odx, Chain.save_nifti and Chain.from_nifti wrap these. Chain.build(cache=...) also accepts an .odx cache path.
  • marginal_transition_metrics computes per-voxel CoDI and CoAsy from a chain.

Branch arbors and flux streamlines (branches.py)

  • extract_branch_arbor traces a bundle's trunk and cortical branches along cross-section ridges of its AFD map.
  • extract_flux_streamlines integrates RK4 streamlines through the bundle's reactive flux field (BundleResult.flux26), seeded along the arbor and across the bundle's AFD support.
  • write_trx, write_arbor(s), write_isosurface_gifti, write_trxviz_project and render_trxviz export the results. render_trxviz finds trxviz-cli through TRXVIZ_CLI or PATH.

Solver (solve.py)

  • segment_bundles(..., on_step=...) calls back with a SolveStep (phase, iteration, residual, solution so far, current wavefront, column signs, and the backward solution during the forward phase) after every iteration. The TRXViz bridge uses this to stream frames; nothing changes when it isn't passed.
  • solve_series and _solve pass on_step through.

TRXViz bridge (trxviz_bridge.py)

python -m mpat.trxviz_bridge {build_transitions,solve_bundle} <request.json> reads a request written by TRXViz. It reports PROGRESS lines and finishes with a RESULT JSON line.

build_transitions accepts four kinds of input:

  • a pyAFQ derivatives folder;
  • a folder or file holding a staged subject .npz;
  • an fODF file (.nii.gz, .odx, .fib.gz);
  • prebuilt transitions (mpat_transitions_<hash>.npz or a transitions .odx).

Other behaviour:

  • Fresh builds are cached as mpat_transitions_<hash>.npz, keyed on the config, max length, fODF, affine and tissue classes.
  • It also returns the seven MITTENS transition maps: CoDI (double/single), CoAsy, P(terminate) (double/single), total amplitude and single-ODF KL divergence.
  • Prebuilt transitions keep the settings they were built with. A request that asks for different settings, or for tissue-mask overrides, is refused with a message naming the mismatch rather than silently ignored. For a transitions .odx, the maps are computed from the fODF stored in the file, which covers white-matter voxels only.
  • An fODF with no tissue maps (pyAFQ pve_internal, desc-mpat_dseg, or a staged .npz) needs a transient white-matter mask from the caller; tissue classes are not guessed from the fODF.

solve_bundle applies TRXViz's ROIs to the cached chain:

  • include, exclude, start and end ROIs, a limiting mask, an optional 26-connected seal, and midline exclusion.
  • Midline exclusion picks the side with afq._is_left: from the name, else from the ROI centres.
  • Ordinary bundles are solved with segment_bundles, the same solver mpat run uses. A small custom solver remains only for what mpat.solve can't express: absorption inside white matter (a term_mask, or start/end ROIs that touch no valid absorbing voxel) and no-end masks.
  • While solving, it writes live density frames to a double-buffered memory-mapped file and prints a STREAM_FRAME line for each, so TRXViz can animate the backward and forward passes.
  • Afterwards it quantifies the bundle (quantify_bundle), extracts the branch arbor and flux streamlines into one TRX, and writes the AFD, density, TDI, harmonic coordinate, coherence, branching and endpoint maps. Extraction failures are reported in the result's warnings and summary.

Smaller changes

  • fields.endpoint_patches and afq.tissue_labels fall back to SciPy when scikit-image can't be imported.
  • kernel.py and the package __init__ default NUMBA_THREADING_LAYER to workqueue.
  • The package exports the new branch, transitions and TRXViz-project helpers.

Behaviour notes for reviewers

  • AFD share is unchanged from main. afd_tract_metrics still divides bundle visitation by whole-brain visitation. An earlier version of this branch regularised the denominator, and that was reverted.
  • Saved transitions use MPAT's own format. The ODX/NIfTI files are written and read by mpat.transitions; they don't go through MITTENS's TransitionResult I/O.
  • Dependencies. odx comes in through mittens-dmri, which requires odx>=0.3.2. Nothing reaches for a local odx-rs checkout or adds to sys.path.

Testing

  • pytest tests: 51 passed, 2 skipped (Python 3.12, odx 0.3.3 from PyPI, mittens-dmri 2.0.0a1).
  • New tests:
    • tests/test_transitions_io.py covers ODX and NIfTI transition round trips, and loading a chain from them.
    • tests/test_trxviz_bridge.py builds transitions on a synthetic phantom, then checks:
      • hash-cache hits and direct loading;
      • refusal of mismatched settings;
      • solves with include, avoid and interior absorption ROIs, and live-frame streaming;
      • flux-streamline extraction;
      • skipping a non-subject .npz in a folder;
      • the error for an fODF without tissue maps.
  • Tested against a real subject (left arcuate; transitions saved by mpat run --ptt --save-transitions odx), with the same ROIs driven through solve_bundle from TRXViz:
    • Bundle density correlates 0.999999 with mpat run's without live streaming, and 0.999997 with it. Total density is within 0.2%, inside the 1e-5 solver tolerance: mpat run solved the arcuate grouped with another bundle, so it stops at a slightly different point.
    • The result has 500 flux streamlines and 7 arbor branches, with no extraction warnings.
    • The seven transition maps from the .odx are distinct maps, and a request for a different angle than the file was built with is refused.

- run() and the CLI gain a --ptt preset plus explicit --step, --stop-mass,
  --fod-thresh, --fod-power, --curv-sigma and --probe-radius; --sift2
  calibrates seed launches; --arbor writes per-bundle branch arbors; and
  save_transitions persists transition probabilities ("odx", "nifti" or
  "both").
- transitions.py saves and loads transition probabilities as ODX or 4-D
  NIfTI and computes marginal transition metrics (CoDI, CoAsy).
- branches.py extracts branch-arbor ridges and reactive-flux streamlines,
  writes them as TRX, and exports and renders TRXViz projects.
- solve_series/_solve take an on_step callback run after each iteration.
build_transitions and solve_bundle subcommands read a TRXViz request JSON,
report PROGRESS lines, and stream live density frames during the solve
through a double-buffered memory map (STREAM_FRAME), so TRXViz can show the
bundle forming. Transition caches are mpat_transitions_<hash>.npz. Flux
streamlines and branch arbors are written as TRX, and extraction failures
are reported in the result instead of leaving an empty bundle.
trxviz_bridge no longer puts a local odx-rs/python on sys.path ahead of the
installed odx, and render_trxviz finds trxviz-cli through TRXVIZ_CLI or PATH
only. Install odx from PyPI, like mittens-dmri already requires.
- afd_tract_metrics divides bundle visitation by whole-brain visitation
  again (n_b / n_w, as on main). The data-dependent regulariser changed
  every bundle's AFD and stopped the shares from partitioning each voxel.
- segment_bundles takes on_step(SolveStep), called after every backward and
  forward iteration, so callers can stream a solve without their own solver.
- Transitions ODX files record the maximum length they were built for, and
  mpat run passes it.
- Import find_boundaries for folders with PVE tissue maps.
- Pick a folder's staged subject .npz by its arrays, not its file name, so
  chain and transition caches are skipped.
- Without tissue maps, require a transient WM mask instead of guessing
  tissue classes from the fODF.
- Prebuilt transitions (.odx or cached .npz) refuse requests for other
  settings or for mask overrides, and load errors are raised instead of
  falling back to a rebuild.
- Transitions from an .odx get all seven MITTENS maps, computed from the
  stored fODF, instead of stand-ins.
- Midline exclusion picks the side with mpat's _is_left (name, else ROI
  centres) instead of matching "_l"/"_r" anywhere in the name.
- Ordinary bundles are solved with segment_bundles, the solver mpat run
  uses; the custom solver remains only for interior absorption and no-end
  masks. Frame streaming is shared by both.
- The precompute command's PTT defaults match mpat run --ptt.
Keep both sides' new run() options: main's viewer colormap (viz_colormap,
--colormap) and this branch's PTT, arbor, SIFT2 and save-transitions options.
@36000

36000 commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

sorry I undercut this with some ipyniivue changes, if its ready to merge, I can rebase and merge for you

@36000

36000 commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

I also pushed one last commit directly to main just now changing default colormap from turbo to actc, i just really wanted to make that change ASAP so people's first experience with the software wouldn't be those ugly isosurfaces

Brings in the actc default colour map.
@mattcieslak

Copy link
Copy Markdown
Collaborator Author

this should be good to go!

@36000
36000 merged commit a1aac12 into main Nov 17, 2026
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants