Skip to content

Fix zero-geometry generation; optimize scoped input preparation (#172, #132) - #206

Merged
trissim merged 23 commits into
mainfrom
fix/input-preparation-20260929
Oct 1, 2026
Merged

trissim merged 23 commits into
mainfrom
fix/input-preparation-20260929

Conversation

@trissim

@trissim trissim commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Validated paired input-preparation checkpoint

Parent integration owner. Existing PR206 normally includes current main74eac059
and typed namespace worker328; paired PolyStore16 production91fd7e854 is adopted.
Metadata ownership stays on existing frozen declarations and backend state, with
MI metadata detection and declaration-derived registry behavior. No singleton
mutation, duplicate reader/store, fallback, assertion weakening or guard waiver.

Current29 source checks pass in both import orders; five namespace controls also
pass with independent custom filenames. Original packaged structural guards pass
with zero positive deltas for application, benchmark and scripts. Earlier125
quality controls and ROI work-count/profile are historical evidence at their
recorded source, not additional current tests or full NRA/ZIP acceptance.

Actual isolated paired wheels import from their installed environment. Persistent
installed MCP health/reopen/pixel samples succeed with generic dependency filename
independent of OpenHCS metadata. Raw ImageXpress generation at zero geometry then
real initialization/compile and persisted virtual sampling succeeds. Empty-step
compile tests initialization only, not a processing or biological pass.

Raw singleton Opera generation succeeds but initialization fails grid inference;
Averroes owns its focused follow-up under issue172. Original persisted Opera also
has A01/R01C01 well disagreement. Keep172 open; real Java/CZI/native execution and
full NRA/R1/ZIP acceptance remain follow-ups. No hosted CI waiting gate or public
registry/global-live readiness claim. Original failures and aggregate CLI exit2
remain preserved; final same-process health passed and all owned PIDs exited.

Canonical receipt: docs/validation/input_preparation_installed_20261001.rst.
Refs #327 and issue172, not automatic
closure of unfinished acceptance. Parent merges the useful validated checkpoint
and continues the remaining scope.

Fixes #327.

@trissim

trissim commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

Architectural review: finish the owner boundaries for discovery filtering and reader lifecycle

Reviewed 006361ff28528c0159bed0999149c71b8befda94, pinned PolyStore f94bbbe8631a4c78f7ceaec72fb99671b58c9a18, and paired PolyStore #16 (8c886e7).

Two production ownership seams still need attention:

  1. Physical binding selection is authored in two consumers. New SourceBindingsConfig.discovery_path_matches (source_bindings.py:1854–1872) reconstructs explicit_source.resolved(root).uri equality and loops over binding.selector.filters. SourceBindingWorkspaceProjector.candidate_matches_binding (source_binding_workspace.py:973–989) independently implements those same decisions after decode. Keeping predecode possible membership distinct from final metadata/component membership is correct; duplicating their shared physical-path decision is not. A change to explicit-source identity or selector path policy requires synchronized edits in both.

    Put physical-path admission on the existing NamedSourceBinding/SourceSelector contract, reusing the original SourceFilterMatcher and SourceFilterTargetResolver families. Both discovery and final projection should invoke that owner; final projection additionally checks decoded metadata/components. Preserve companion-path provenance and unrestricted-binding union behavior rather than forcing predecode to guess metadata. A new path-selector behavior should be one matcher/owner extension, not another pair of consumer edits (IMPL-12 / BOUND-2).

  2. The new Java probe reconstructs a dependency-owned lifecycle. BioFormatsPlaneStoreAdapter._is_single_file, lines 827–841 calls ensure_initialized, constructs context.ImageReader(), invokes Java and closes the reader itself. The pinned polystore.bioformats_java.BioFormatsJavaContext already owns reader construction, initialization, failure cleanup and the related declares_path probe. Add the single-file capability there and consume its public operation from OpenHCS. The source-selection policy remains OpenHCS-owned; Java reader lifetime does not. Moving this body to another OpenHCS helper would preserve the duplicated mechanism (IMPL-13 / BOUND-2).

The paired ROI change is a good example of the intended direction: TwoDimensionalLabeledMaskROIExtractor uses scikit-image's already-owned RegionProperties.image/bbox instead of rescanning/reconstructing them. FijiROIConverter.rois_to_imagej_members still legitimately projects each parent's shapes to individual external ZIP members; those are not additional biological identities. Keep the recorded gitlink/paired-validation gate explicit so the OpenHCS PR does not claim the upstream optimization before adopting that validated revision.

These requests concern production owners, not fixture/test style. The existing 16-container, companion-source and raster/contour regressions are useful behavioral controls to retain through the factoring.

@trissim

trissim commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

Both production findings are addressed in the existing paired drafts, not just fixture/lint changes:

  1. IMPL-12 / BOUND-2, OpenHCS commit32a2b27e2: NamedSourceBinding.physical_path_matches owns explicit-source resolution/equality; SourceSelector.path_filters_match delegates clauses to the original matcher/target-resolver families. Both discovery and final projection call this binding operation. Removed duplicated physical procedures/consumer filter loops. Metadata/components still require decoding; config retains global filters and alias union. Companion filters retain store provenance but do not replace exact entrypoint identity.
  2. IMPL-13 / BOUND-2, PolyStore commit0aa057b in draft16: BioFormatsJavaContext.is_single_file and existing declares_path share context-owned _probe_reader initialization/construction/unconditional close. Deleted OpenHCS's helper; its adapter invokes the public capability. No metadata/setId open in the probe, new context/helper, companion roster or cache. Existing in-place ROI optimization is unchanged.

Executed 28 lightweight passes in0.50s/72.8 MiB peak RSS:17 parent physical-admission cases/guard;11 child Java-context cases/guard. Exact relative/absolute/file-URI identity, directory-regex/OR groups, metadata/component deferral, decoded companions, unrestricted alias union, true/false/failed probe cleanup/retry, existing context cleanup. Controlled external responses, actual owner operations. AST guards prohibit the reviewed bypasses; seven touched Python files parse; new tests/touched child source pass Ruff, both diffs pass whitespace checks.

This is not a resolution of the retained141 passed/26 failed combined-suite result. Separately bootstrapped application preservation, broader sixteen-container/AUTO/TIFF rerun, bounded after-profile (not run), real Java/CZI and installed acceptance remain open. Recorded gitlink remainsf94bbbe; parent now requires the published paired capability before coordinator adoption. Explicit source imports verified; no MCP/JVM/GUI, shared lock takeover, installation/skill/merge or blind runtime/data changes. Guard11.4 GiB available, final11.2 GiB; tiny owned92 KiB scratch removed.

Full updated evidence/boundaries are in both PR bodies and durable receipts: parent6ffea8d61 / childa14a4fc. Lovelace retains ownership of the remaining source/preservation/profile checks; runtime depends on the next finite coordinator handoff.

@trissim

trissim commented Sep 29, 2026

Copy link
Copy Markdown
Collaborator Author

Authorized critical cache/progress integration checkpoint is published as OpenHCS228 at bf9aae4620585e98d10e469037ef98237884b4f8, paired with PolyStore17 at c16b8fc99a07f17e92e5fbf46f312d73cc9c46d7. Both remote heads verified; exact child gitlink adopted by228. Base current main213439a04 includes merged218. References224,206 and16; original branches/source/receipts preserved, no force/rebase/reset.

This extracts the already-owned cache/progress implementation, not another runtime/materializer. Source-selection/Java probe/ROI optimization stays in206/16. Their 141-pass/26-failure receipt remains unresolved; after-ROI profile did not run. No claim of broad preservation or installed acceptance from this checkpoint.

Before application/MCP startup set POLYSTORE_IMAGEJ_CACHE_ROOT=/home/ts/.cache/polystore/imagej (bundle parent, not hashed directory) and POLYSTORE_IMAGEJ_ALLOW_DOWNLOAD=false. Both settings belong to existing PolyStore distribution/download-policy owners, projected by existing OpenHCS launch owner. Independent XDG logs/caches remain independent. Existing progress/worker binding stays canonical.

Current-main pair: 36 passed5.14s, peakRSS277712KiB, availableRAM14.4→13.96GiB. Two fresh native processes with production download-forbidden policy, separate XDG roots, actual same existing Fiji/Java paths, zero new cache entries/no downloads/no JVM. Reproducer and durable receipt are in228. No override/copy/archive in native proof, no installed/frozen/scientific change or replay. Parent owns review/merge/install and resource-coordinated real installed stdio inspection + concurrent discovery/heartbeat. No Closes/readiness claim until that live gate.

…tion-20260929

# Conflicts:
#	openhcs/utils/environment.py
#	tests/unit/agent/test_plate_inspection_cold_runtime.py
@trissim

trissim commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator Author

Parent is taking a narrow newly reproduced acquisition-calibration publication defect through issue302/PR304: a real synthetic HTD raw route uses0.65um but ordinary Gaussian compiled output loses spacing and defaults to1.0. Independent read-only trace identifies save_virtual_workspace_metadata publishing calibration separately from per-source payload metadata. PR304 will preserve native source spacing and publish acquisition-owned calibration through existing typed owners; no edits to this worktree or physical-selection/BioFormats/ROI changes here. Integrate304 normally when continuing206. Original failures and source claims retained; this is not a duplicate input-selection implementation.

@trissim
trissim marked this pull request as ready for review October 1, 2026 03:15
Copilot AI balanced review requested due to automatic review settings October 1, 2026 03:15

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Metadata namespace depends on PolyStore/OpenHCS import order

2 participants