Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
95 changes: 32 additions & 63 deletions research/vestibular_schwannoma/deployment/pacs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,45 +106,18 @@ least 32768 and values outside it lowered to at most 32767: the blue region is e
mask, and yellow marks voxels with some foreground probability outside it. Existing `fused`, `fused_vote_map`, `reports`, `labels`, and `redcap` directories are rejected
to prevent overwrites.

### Recoverable masks in REDCap JSON

The final output now includes `redcap/<mask-series-UID>/output.json` and
`output_data_dictionary.zip`, using pr2mask's EAV row format and instrument.
`redcap` is an owned output directory and must not already exist.
The exporter reads the **written DICOM mask**, matches it to the source images by
geometry, and stores the full binary mask once in the model's permanent `pr2mask` repeat instance.
It also handles an empty measurement list using the source PatientID and
ReferringPhysicianName (`EventName:` prefix removed), matching pr2mask routing.

Added fields: `vs_mask_json` and `vs_measurements_json` (Notes), `vs_model_type`, `vs_bundle_sha256`,
`vs_prediction_id`, `vs_deployment_version`, `vs_tta`, and `vs_created_at`.
The existing data dictionary is extended; measurement definitions and OriginID
are retained. Compression happens in memory, so the mask is JSON text rather
than a separate binary attachment. There is no direct REDCap uploader.

`vs_mask_json` is a JSON string with schema version 1:

- Binary voxels: C-order `(slice, row, column)`, foreground 1, little-order bits,
gzip-compressed and Base64-encoded (`packbits-gzip-base64-v1`).
- Shape, foreground count, and SHA-256 of the decoded C-order uint8 voxel bytes.
- DICOM LPS geometry: `pixel_spacing` in row/column order,
`image_orientation_patient`, and `image_positions_patient` in encoded slice
order. **All lengths are millimetres by schema convention; no units field.**
Slice order follows increasing position along the DICOM orientation normal.
Per-slice positions preserve irregular spacing; SliceThickness is not used
as a substitute for the distance between slice centres.
- Source Study/Series/Frame of Reference and ordered SOP UIDs, mask series UID,
model type, member IDs, bundle hash, deployment version and TTA.
- Stable prediction identity (hash of mask, geometry and provenance) and UTC
export timestamp. The timestamp is excluded from the prediction identity.

Recover voxels with `redcap_output.decode_mask(json.loads(row['value']))` for the
`vs_mask_json` row. For voxel `[s, r, c]`, its LPS position is
`position[s] + r * pixel_spacing[0] * orientation[3:6] +
c * pixel_spacing[1] * orientation[0:3]`.
Use the same spatial grid as the reference before computing Dice. Distance
metrics must respect geometry; irregular grids require physical-coordinate
surface distances or a documented resampling step.
### REDCap output

The final output includes pr2mask's `redcap/<mask-series-UID>/output.json` and
`output_data_dictionary.zip`, in pr2mask's EAV row format and instrument. `redcap` is an
owned output directory and must not already exist. There is no direct REDCap uploader.
The exporter sends only pr2mask's own columns, in the model's permanent `pr2mask` repeat
instance. An empty measurement list is routed with the source PatientID and
ReferringPhysicianName (`EventName:` prefix removed), matching pr2mask routing. The mask
itself goes to the PACS as the `labels` series.

Up to 20261026T134845Z the exporter also sent eight `vs_*` fields, among them the mask as
`vs_mask_json`. They are no longer sent.

### Stable model destinations and overwrites

Expand All @@ -164,30 +137,26 @@ All fields for a model share its repeat instance. Within the same record/event:
- The same bundle overwrites its previous result, including when TTA, deployment
version, timestamp or the predicted mask changes.
- A different bundle uses a different instance, including different weights of
the same architecture. `vs_prediction_id` remains provenance, not a routing key.

pr2mask's per-region measurements are preserved in `vs_measurements_json`, a JSON
list of `{redcap_repeat_instance, field_name, value}` entries. The instance inside
this list is the original **region** number, not the model destination. Regions are
not emitted as separate top-level repeat instances, which would collide with the
model instances. Instead, pr2mask's own measurement columns in the model instance
hold the **largest** region (most voxels; ties keep the lower region number), and the
list holds every region. pr2mask numbers regions in scan order, so the largest is not
always region 1. An empty mask has no regions, so pr2mask lists no columns; the exporter
then takes them from `pr2mask_dictionary.csv` and sends them blank, so the importer must
overwrite blanks to clear values from an earlier run.

Since the Fiona base of 2026-10-19, pr2mask also stores each region's mask in seven
`mask_*` fields. These fill the columns for the largest region but are left out of
`vs_measurements_json`: the mask alone is about 84,000 characters for a 512×512×120 scan,
and REDCap cuts values longer than 65,535 characters, so it only decodes for small volumes.
`vs_mask_json` holds the lossless mask in about 22,000 characters.

`pr2mask_dictionary.csv` holds the `instrument.csv` rows (without the `vs_*` fields) of a
non-empty run of the packaged pr2mask. When the Fiona base changes, regenerate it from a
smoke test and check that a non-empty run lists the same columns.

Install the extended dictionary in the receiving REDCap project and qualify a
the same architecture.

pr2mask writes each region as its own repeat instance (1, 2, 3, ...), which would collide
with the model instances, so regions are not sent as separate instances. pr2mask's columns
in the model instance hold the **largest** region (most voxels; ties keep the lower region
number). pr2mask numbers regions in scan order, so the largest is not always region 1. An
empty mask has no regions, so pr2mask lists no columns; the exporter then takes them from
`pr2mask_dictionary.csv` and sends them blank, so the importer must overwrite blanks to
clear values from an earlier run.

Since the Fiona base of 2026-10-19, pr2mask also stores the mask in seven `mask_*` fields,
which fill the columns for the largest region. `mask_base64_gzip_compressed` is about 84,000
characters for a 512×512×120 scan, and REDCap cuts values longer than 65,535 characters, so
it only decodes for small volumes. The full mask is in the PACS `labels` series.

`pr2mask_dictionary.csv` holds the `instrument.csv` rows of a non-empty run of the packaged
pr2mask. When the Fiona base changes, regenerate it from a smoke test and check that a
non-empty run lists the same columns.

Install pr2mask's dictionary in the receiving REDCap project and qualify a
PACS-to-REDCap round trip before use. The importer must honor the supplied numeric
repeat instance and update existing values. Live importer behavior and field limits
have not been verified. The record/event must already identify the intended case/scan;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@
TTA_FLIP_AXES,
ParallelEnsemblePatchInferenceEngine,
)
from redcap_output import write_redcap_mask, model_repeat_instance
from redcap_output import write_redcap_output, model_repeat_instance


SW_BATCH_SIZE = 1
Expand Down Expand Up @@ -286,7 +286,7 @@ def _run_postprocessing(
check=True,
)
_tag_pr2mask_labels(work_dir, deployment, use_tta=use_tta)
write_redcap_mask(work_dir, input_dir, deployment, version=version, use_tta=use_tta)
write_redcap_output(work_dir, input_dir, deployment)
_copy_final_outputs(work_dir, output_dir)
_publish_log(log_path, output_dir)

Expand Down
Loading
Loading