Skip to content

Preserve source-bearing fractional-Z points in live projection (partial #152) - #494

Closed
trissim wants to merge 215 commits into
mainfrom
fix/live-fractional-z-point-projection-20261002
Closed

trissim wants to merge 215 commits into
mainfrom
fix/live-fractional-z-point-projection-20261002

Conversation

@trissim

@trissim trissim commented Oct 2, 2026 •

Copy link
Copy Markdown
Collaborator

Stack disclosure and receiver split

This published head e6c0bf77f8e4d24d80c0463e214070bc018a3301 is ROOT-DEPENDENT, not a main-only point fix. Its ancestry imports Root394 4eaacf17 and full b312148855b6ed486442d15e9395a09e453d77c6. Against verified remote main f7de9efad393bce83a3756a271f0cd4383783ea9, GitHub shows 157 inherited/owned files (93 production). This PR must NOT merge those unreviewed Root files into main under point-repair acceptance. All previous heads and qualification evidence remain unchanged.

Parent authorizes a separate MAIN-BASED RECEIVER-ONLY draft with the existing five receiver owners, preserving this producer checkpoint and Root435 dependency. The standalone unchanged 989-byte archive receiving journey will use that separate receiver with reviewed511/519, not this full Root stack. No producer execution replay or SCI package mutation. The prior source results below apply to their exact Root-dependent source, not automatically to the new main-based split.

Actual status

Receiver source repair is implemented; full public producer acceptance is not complete. This is partial #152/#134, not closure of all orthogonal/output acceptance.

Latest coherent source is e6c0bf77f (normal main f7de9efad integration). It includes c7a11af98, with native world translation now requiring the owner's scale rather than retaining an optional unscaled branch. Seven foreign dirty gitlinks are preserved and excluded from authored commits.

What changed

  • The original typed PointROI producer retains represented source planes and an exact first-source anchor through original materialization/live stream declarations.
  • Persisted StreamingService.stream_rois now transports the original source-bound ROI. ROIArchiveSourceMetadata owns the single feature-only source exclusion reused by native Points and Shapes; reopening no longer discards the transported source record.
  • Native fractional-Z Points retains the represented source span from its anchor, not only anchor through the occupied point's ceiling. Existing missing-Z/anchor/fractional/domain guards remain.
  • NapariAxisPresentation derives scale, units and world translation together for Image/Shapes/Points and shared-axis reprojection. Declared three-dimensional spacing calibrates Z; two-dimensional/unspecified spacing does not invent Z, and selectors/bands remain dimensionless.

No alternate codec, registry, axis-domain mirror, source-provenance reconstruction or Root publication/compilation edit. Singer #522 confirms these native methods are disjoint from his retirement lifecycle (#522 comment5966729112).

Real failed journey and exact remaining boundaries

Parent-owned engineering95 original public compile completed, but execute 9292bfb8-4d2e-4e54-9bcc-5af998083fe5 FAILED at project_runtime_artifacts with conflicting metadata for a persisted image occurrence, after points were saved/streamed. Root #435/#394 owns that join. The actual conflicting address/metadata pair is not logged; no speculative guard weakening or writer replay was performed. Owner request: #394 (comment)

Original PolyStore readback of the actual 989-byte ZIP proves all four original source planes, fractional Z1.5/object7 and2/.65/.65 micrometer spacing intact. Archive SHA256 fd0a372f3100c73da9533b308d44c2f85fa7a86b733e2608ae986135cb4e401a. The old public reopen dropped the source record, narrowed route Z0..3 to0..2 and displayed Z scale1. All9 original XY/XZ/YZ PNGs were personally opened: rendering is real, calibrated/source-preserving acceptance is not claimed.

Qualification at its actual strength

  • Prior ordinary target02/03 whole package and paired dependency qualification remain frozen at their original source; they do not contain this newest receiver repair.
  • Final coherent source: 59 passed,8.26s,279,625,728-byte peak,512MiB/noSwap/oneCPU,OOM0. Original failed direct-source import and the test's incorrect pre-writer comparison are retained. The corrected assertion compares entire original saved metadata against entire reopened metadata and original plane provenance; no failing field is removed.
  • Original writer/ZIP/source guard regression family plus native feature and2D/3D/relative/unspecified world-transform controls. Real Napari Image/Points coordinate transforms and Dims semantic navigation exercised without a Qt application/server.
  • Before/after full named AST census:704 production+95 actual dependency modules,zero parse omissions. Existing owner/consumer family and removed native translation copies recorded; this is source evidence, not a full NRA equivalence proof.

Evidence: docs/validation/live-fractional-z-point-projection-152.rst; persistent engineering494/RECEIVER11-QUALIFICATION.rst, original reader JSON, before/final AST inventories, source control logs and parent public journals/captures. No scientist contact, installed hot patch or new runtime was made by this repair.

Next parent-authorized step is one ordinary qualified whole target, coordinated with511/519 owners first, then a distinct new public reopen-only context using the unchanged original archive. Automatic producer success still depends on Root's publication repair; no CI wait and no failed/UNKNOWN operation replay.

trissim added 30 commits October 1, 2026 13:55
Move pure input-relation resolution to ArtifactSpecCollection and canonical output contextualization to RuntimeReturnedOutputMatcher. Delete duplicate CP executor helpers and retain composed measurement aliases from their declared input source.

Refs #384. Validation: 454 focused artifact/output/CellProfiler tests pass; fresh import smoke and diff checks pass.
@trissim

trissim commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator Author

Guide-only #512 is independently MERGED on main d110322. Normal main integration published here at9487b8da19dd; tracked-tree diff vs04ce7edb0 is empty (same reviewed guide bytes), five foreign dirty gitlinks preserved. Source07 qualification remains at its original strength: unchanged31=25PASS/6FAIL; three added identity controls2PASS/1FAIL. Root exact method dependency5965533217/5965551217 remains; no unchanged retest, source patch, wheel/build/runtime or active-scientist sync.

@trissim

trissim commented Oct 3, 2026

Copy link
Copy Markdown
Collaborator Author

494 public95 failure: original archive and receiving owners

Planck owns494. Original job-2/execution
9292bfb8-4d2e-4e54-9bcc-5af998083fe5 remains definitively FAILED.
No native/client/viewer call or scientific replay was made by this diagnosis.
Source7a067c505444ec7d8720da21118dc85787129c59; installed immutabletarget03
contains qualified bd6d OpenHCS production. Root current30ebacf907273ee0d0767f392ac21281915f5a0e
differs from inspected60b2 only benchmark/docs, not determining owners.

Actual saved archive is intact

Original PolyStore load_rois_from_zip, ROIArchiveSourceMetadata.decode and
ROIFractionalZ.source_component_domain read the actual989B archive:
fd0a372f3100c73da9533b308d44c2f85fa7a86b733e2608ae986135cb4e401a.
It contains one point, exact center_z1.5/center_y1.5/center_x2.5, label/object7,
all four ordered original source identities Z0..3 with unmodified nested raw
records and calibration SourceVoxelSpacing((2,.65,.65), MICROMETERS).
ARCHIVE95-ORIGINAL-READER09.json contains complete original decoded metadata,
guarded domain and original geometry/item-field projections, not a replacement
ZIP decoder or handcrafted source fixture. Original bounded helper terminal0,
512MiB/noSwap/CPU1/60s; no source/artifact edits. Raw archive remains immutable.

Persisted reopen loses the full source declaration

ViewerStreamingCoordinator.stream_rois reads/validates original metadata and
derives exact domain; then data_list.append(ROIArchiveSourceMetadata.geometry(rois))
removes openhcs_image_payload_metadata before transport.
StreamImagePayloadMetadataProjector.item_fields sends declared image_metadata
spacing/spatial/axis fields only; ImagePayloadMetadata.viewer_field_names excludes
source_provenance. Thus full source identity is absent from the reopened public
point payload although its saved bytes and original live point have it.
The item-fields route DOES retain explicit2/.65/.65 spacing; this is not proof
that native Z scale honors it, nor that full source identities survive reopening.
Do not infer calibration from a shared raw layer's transform.

Required original-owner closure: preserve bound ROI payload across live/reopen;
filter source-only records only at existing native feature projection. Existing
_build_nd_points already excludes ROIArchiveSourceMetadata.FIELD from features;
NapariShapeFeatureColumns must share the source owner's feature projection,
not stringify transport provenance into a feature column. No extra sidecar,
registry, codec, metadata reconstruction, source-path override or weakened guard.
Literal FileRoiSidecarArtifactSourceProjection is not found in this exact installed
production, currentmain or fetched Root394; the actual owner here is
ROIArchiveSourceMetadata plus ViewerStreamingCoordinator. This does not prove
that a newer unpublished owner does not exist.

Native Z domain and spacing are independent receiving defects

NapariPointsLayerDisplayHandler.geometric_component_values validates a fractional
point against declared Z0..3 but returns only occupied anchor..ceil(point),
Z0..2 for1.5. ViewerLayerAxisProjectionRequestAuthority uses domain.observed()
for the route rather than all declared viewer values. Required correction is
point-source represented coverage through that existing handler, not making all
viewer routes dense or fabricating source planes; anchor/out-of-range guards stay.

NapariAxisPresentation.spatial_layer_kwargs assigns all component-axis scales1
and units dimensionless, using source spacing only for XY. Explicit three-axis
calibration is discarded for projected Z. A coherent correction must include
translation/reprojection and native navigation/label-domain consumers, not only
change one layer.scale call. Two-axis/unspecified source calibration must NOT
invent Z; other acquisition component axes remain dimensionless. Preserve raw
physical pixels, fractional native coordinates, source provenance and strict
foreign-domain/duplicate/order/calibration rejection.

Source publication remains Root435/394, not a point receiver workaround

Native trace shows point archive written/streamed before metadata publication
fails in steps/function_outputs.project_runtime_artifacts, after write() calls
produced_projection_metadata. That guard compares produced SourcePlaneProjection
address AND image_metadata against the saved Output projection. Neither actual
conflicting pair is printed/exported; runtime-values95-01.json was not produced.
The precise differing field is UNPROVEN. Current Root determining guard is
unchanged. Existing issue435 names duplicate metadata publication; issue516
instead concerns FilterObjects lineage and must not be used as this diagnosis.
Root owns measurement/correction of its full persisted-occurrence join; retain
strict genuine-conflict rejection, no delete/retry/stream-disable to hide it.

Coverage, claims and acceptance

PRESENTATION-FAMILY09.json reuses original refactor-audit ParsedModule/Repository
scanner: all704 production+95 actual paired PolyStore/ZMQ modules parsed with
zero omissions. Named owner/bases/behavior/import/consumer evidence; dynamic
MRO/source relationships read semantically, NOT full NRA behavioral proof.
BOUND-2: preserve original typed source owner at boundary rather than lose then
reconstruct. IDEN-1: represented source extent is not occupied point extent.

Current394 changed files exclude native viewer and core viewer_streaming_service;
407/413 typed summary source change already merged. Singer522 independently owns
route retirement in napari_streaming_handlers/viewer_server, so coordinate exact
presentation/point/feature methods, not overlapping its lifecycle changes.

Personally opened all9 existing PNGs: firstXY triple and reopened genuineXZ/YZ
triples. Green point is centered over the small white source patch but marker
size obscures much of it; screenshots prove rendered transport, not correct
Z-domain, source preservation or world calibration. Their original public
state/payload observations remain authoritative. Parent's partial reopening
errors[] is transport success only. Entire494 automatic producer lifecycle and
correctly calibrated persisted reopening are NOT accepted. All failed attempts,
original journals/source/targets and current scientist custody remain protected.

@trissim

trissim commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Cross-machine production-freeze handoff (5 Oct)
The opening body and old receiving failures are not the current integration state: the later #748 native source/geometry integration is already merged and has its own receiving evidence. Please do not apply this older full branch over main as a fresh geometry fix without comparing the determining production diff.

Planck is the existing source/receiving owner and has been asked to reconcile this PR against merged main, close superseded work, and name any genuinely remaining partial #152 requirement. Existing failed attempts remain preserved; they are not requests to replay. This draft is not a separate ready production patch or a reason to postpone a main freeze.

@trissim

trissim commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator Author

Owner reconciliation for the production-freeze sweep: this historical implementation stack is superseded, not a pending merge candidate.

Root #394, the main-based receiver #524 and automatic point/scalar delivery #748 are merged. The determining current delivery receipt is automatic-point-domain-152-20261005.rst, with its completed-delivery appendix on current main. Original receiving15 execution 8c48fe80-a0de-4883-9d1e-3b68527afba9 completed with errors[]; automatic settlement and same-viewer saved scalar TIFF, fractional Points and plane-anchored Shapes reopening were verified. Full Z0..3, fractional centre, calibration, scalar values and native placement were checked; all14 original captures were personally opened. Parent pinned acceptance is #748 comment5998423718.

The original failed9292bfb8 attempt and989B archive remain immutable failures/evidence, not retroactively successful or replay requests. The old Root-seam hold is obsolete. Closing this superseded branch without merging its historical Root ancestry or deleting its branch. This does not close all #152 obligations, the distinct source-bearing graph reopen boundary in #404, or #541's graph-unit acceptance.

@trissim trissim closed this Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant