Add stitch refinement and raw-tile illumination correction worker - #174
Conversation
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5a6d4002fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| position_index = frame // frames_per_position | ||
| source_position_indices.append(position_index) |
There was a problem hiding this comment.
For an ND2 whose flattened sequence is time-major (or otherwise not P-major), frame // frames_per_position groups frames from different stage positions together; the subsequent consistency check then rejects a valid P×T composite, or associates frames with the wrong translation if their coordinates happen to match. The source records expose axis metadata such as xySet, and the ND2 supplies loop_indices, so derive and validate the position from those explicit indices rather than assuming a P→T→Z→C flattening order. This is especially important because the worker documentation advertises preservation of P×T data rather than limiting support to P-major files.
AGENTS.md reference: AGENTS.md:L49-L57
Useful? React with 👍 / 👎.
|
Fixed in The parser now validates the explicit P×T×Z Added a time-major P×T×C regression test using TDD. The branch-wide audit found no other production instances of flattened frame-to-position inference; one P-major test fixture was rewritten. All 19 Docker tests, flake8, the real Well_2 metadata check, and GitHub CI pass. @codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f9d97f8afe
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ) | ||
| stage_positions = [] | ||
| for position in range(positions): | ||
| position_stack = np.asarray(raw_file.asarray(position)) |
There was a problem hiding this comment.
For a time-lapse ND2, raw_file.asarray(position) materializes the entire (T, Z, C, Y, X) stack even though line 371 immediately discards every time point except the first. Memory therefore scales with the full time series per position; a 100-time-point, 6-Z, 5-channel 2048×2048 uint16 acquisition requires roughly 25 GB just for this temporary array and can OOM before correction begins. Read only the T=0 sequences needed for the reference and model planes instead.
Useful? React with 👍 / 👎.
| output_positions = ( | ||
| refinement.positions | ||
| if settings["refine"] | ||
| else np.rint(layout.positions).astype(np.int64) | ||
| ) |
There was a problem hiding this comment.
When a deployed multi-source document contains fractional x or y translations, clearing Refine stitch positions still rounds every position here, after which corrected_source_document() writes the rounded integers. The supposedly unrefined output can therefore move each tile by up to half a pixel and no longer retains the deployed geometry promised by the worker documentation; preserve the original translations without integer conversion in this path.
AGENTS.md reference: AGENTS.md:L49-L57
Useful? React with 👍 / 👎.
|
Fixed both findings in T=0 training memory: Fractional geometry: clearing refinement now copies the deployed translations without Branch audits: the full-stack-read pattern had no other production instances. The geometry-coercion sweep found and fixed the downstream x/y Verification: 22/22 Docker tests, flake8, GitHub @codex review |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…analysis (#24) * Document Stitch Refinement + Illumination Correction tool Adds documentation for the new image-processing worker that refines tile positions and corrects raw-tile illumination on composited .nd2 datasets. - image-processing.md: new section covering the Composite upload prerequisite, workflow, parameters, technical details, tuning, and limitations (including that existing objects are not moved) - images-datasets-and-collections/README.md and file-formats.md: cross-references from the compositing/.nd2 guidance - new-features.md: August 2026 changelog entry Source: arjunrajlaboratory/ImageAnalysisProject#174 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * Document August 2026 features: connections, tracking, analysis, and fixes Documents the August work across both repos, and fixes several pre-existing defects found along the way. New features: - tools-for-connecting-objects.md: new "Browsing and editing connections" section — Connections tab (Flat / By track), scope selector, deletion, Connect selected, track ID property labeling with staleness badges, track-metric filters, and dangling-connection cleanup - analysis-plots-and-gating.md (new page): Analysis panel, lasso gating, chained gating strategies, server-side gating at scale - time-lapse-mode.md: Time Lapse palette moved out of the Navigator, track coloring, "N tracks · M links" readout, clickable connection lines - interacting-with-objects.md: Measurements tab, chip strip, and Color by Property - large-annotation-datasets.md: zoomed-out raster overview and the active-constraint cue on the count indicator Worker changes: - Cellpose-SAM Diameter corrected to default 30, range 10-200 - `all` accepted in batch coordinate and Crop range fields Screenshots (captured live against localhost:5173): - timelapse-mode-panel.png replaced; the old shot showed the controls inside the Navigator, which contradicted the new text - connections-by-track.png and analysis-scatter-plot.png added Pre-existing fixes: - three broken relative links (../documentation/... from inside documentation/) - stray drafting sentence left in tools-for-connecting-objects.md Sources: NimbusImage #1298 #1302 #1311 #1312 #1327 #1332 #1333 #1339 #1340 #1341; ImageAnalysisProject #156 #170 #171 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 participant
Summary
multi-source2.jsonlarge_image_converter, and upload a new pyramidal TIFF with complete Girder provenancepyvips/libvipsmismatchThis branch starts fresh from
masteras requested. It implements the raw-ND2 workflow and intentionally does not implement the stitched-image/light-mode fallback from #172; no frontend files or compositing geometry were changed.Review fixes included
limit=0pagination pathpyvips/libvipswheel after reproducing a converter segmentation faultsendErrorValidation
docker run --rm annotations/illumination_correction:test— 15 passedflake8 --ignore=E203,E501,W503 ...— passedpython3 -m py_compile workers/annotations/illumination_correction/*.py— passedgit diff --check— passedisGPUWorker=falseWell_2 end to end:
6a94a3b2d233ea7fbc4329b0(0, 920), measured(-4, 930), NCC0.933Coordinate-stability note
The validated Well_2 measurements expose a conflict in the written acceptance criteria: satisfying all 84 measured edges to ≤1.414 px accumulates to a maximum per-position shift of 33 px and outer-bound changes
[-29, -32, +28, +20]px. It is mathematically impossible to preserve all four bounds within ±16 px while retaining the specified ≤2 px edge residual for this dataset. The worker therefore records the full values and emits a prominent warning rather than discarding the validated alignment; existing annotations are not moved.