Repository navigation
Export requested source identity from typed measurement provenance (#731) - #734
Conversation
…/spreadsheet-source-provenance-731
|
Independent parent source review at7300be4: read complete PR diff and determining image-number projection; verified production bytes unchanged from271229f AST/source-qualified checkpoint. Existing CellProfilerDatabaseColumnDialect now owns source-file/extraction/acquisition formatting and both CPA and spreadsheet consume it; old CPA mapping is deleted. SourceMetadataFields and SourceImageProvenance remain actual identity/domain owners. Axisless selection is carried by existing CellProfilerImageSetNumbering rather than another numbering registry, preserving ambiguity rejection and artifact-summary anchoring. Search confirms the related new helpers have their declared single owners and expected consumers. Existing complete production-symbol evidence reports zero parse omissions; this review is focused semantic source review, not a fresh global NRA R1 proof. I found no source-level reason to delay this family beyond the already-running installed public export acceptance. No new tests or package rebuild requested; no hosted CI hold. |
Summary
Fixes #731. Spreadsheet flags already reached the exporter, but copying existing
Image fields could not export filenames or acquisition coordinates never
projected from the typed measurement provenance.
The existing
CellProfilerDatabaseColumnDialectnow owns metadata/file-valueprojection. CPA consumes the same methods; its replaced path/metadata mapping
is deleted. Spreadsheet export consumes producer-owned per-plane identities,
the existing named-source selection and image-set numbering. It does not reload
pixels, guess source names, change grouping or synthesize absent components.
Original extracted metadata keeps its spelling/value and its existing
participation rule (only actual extraction records). Requested acquisition facts
are folded separately using the existing source consensus. Both new projections
are driven by the original flags: false flags do not synthesize extra columns;
already declared upstream Image features are always retained. Conflicting file values
still go through the original sparse accumulator conflict checks.
Owner and consumer closure
Production changes:
database_column_dialect.py, CPAanalyst_export.py:collect_image_provenance, and spreadsheet_measurement_tables/_source_metadata_measurement_rows.Existing
SourceMetadataFields.component_domains/original_itemsremain thesource-field authority. Existing
SourceImageProvenance.for_source_planeandfor_source_imageremain the plane/name authority. No runtime-export schema,materialization, stream or measurement producer changes; Singer owns #728/#732.
Static family evidence is retained at
/home/ts/wt/openhcs-issue-batch-20260929/engineering722/EXPORT-FAMILY-BEFORE.json.All matching tracked production Python parsed; dynamic receiver resolution was
read semantically, not claimed proved by AST. IMPL-12/BOUND-2: remove the separate
CPA mapping and make both consumers use the existing dialect and metadata owner.
Qualification state
68 spreadsheet/CPA controls PASS in original
export-controls06.log/.time,handle25254 terminal0, 10.24s whole process, peak421648KiB, swaps0. Includes a tiny object
table with no upstream Image features, all four flag combinations for both explicit/axisless rows, both
independent contributor orders, absent axes, preserved original extraction path
templates and conflicting supplied filenames rejected. All pre-existing exact
CSV/metadata assertions remain unchanged and pass.
Original negatives are retained: controls01 used the old receiving09 native CSV
ABI (5 arguments, current source requires6); controls02 refused the old ignored
checkout extension origin before tests. controls03 used the correct receiving10
native ABI, passed50 and exposed12 role/default-projection regressions. Corrected
the source role fold and tied extra projection to existing requested flags, not
weakened tests or native compatibility. controls04/05 corrected64PASS remain
preserved. Final source checkpoint271229f93 includes the original numbering
owner's reusable declared-source slice selection: no axisless provenance loss,
no new grid/scope, ambiguity rejection and original summary anchor unchanged.
No source edit occurred after the68-control run; the commit packages those bytes.
Family final evidence:
engineering722/EXPORT-FAMILY-FINAL.json; no parseomissions, dynamic receiver limitation remains explicit. Foreign gitlinks and
untracked histories were not staged or changed.
Actual ordinary installed/public MCP acceptance
Whole receiving11 source7300be4d190c225ad7d5b38d8307c5565faaed34 contains the
exact reviewed271 production bytes, installed from ordinary wheel
d23fcf2ef93a7d764a938f07d9d92366b1eeab94a424074350454bc74524805e.Whole proof:913 RECORD entries,813 tracked source files,790 Python files,
13 packaged/synced skill files. No installed overlay or current scientist change.
Original persistent MCP client7585/native36406 used the existing registered
primary-object/intensity/PLATE spreadsheet declarations, three immutable tiny
12x15 synthetic images, no custom registration or viewer.
Forward session2 job4/16a2f990-5d6d-4e2c-88ae-4390f622fc8f and reverse session3
job6/78829037-80a7-4e71-ae83-90e099c9b32e each COMPLETE/errors[].
All four actual Image/Cells CSVs reconcile one row, declared WellA01/site1/Z1/time1,
blank varying channel, exact producer-named filenames/paths and independent
intensity values430/255 and889/255 (means divided by12). Both orders give identical
measurement values. Original input and complete PipelineDocument hashes unchanged.
Explicit remaining upstream limit: the photometry producer retains only the
first selected image's source identity, despite correct numerical features for
both images. Forward export represents Mask+Process; reverse represents
Mask+Nuclear. These are original virtual workspace filenames with parent
.;this exporter does not fabricate a missing contributor or override it with an
assumed physical path. Singer accepted separate producer triage. This limit is
not represented as complete multi-image provenance or a biological/count claim.
Original session1/job2 STEP_INPUT selector failure is retained; corrected source03
uses declared PIPELINE_START bindings/new destinations, no guard relaxation or
failed-job replay. Local helper02's wrong enum-value assertion and the reverse
CLI's undeclared compile spelling remain explicit predispatch negatives.
Actual receipts:
/home/ts/wt/openhcs-issue-batch-20260929/engineering-pre-first-routing-20261004/receiving11/public88/ACTUAL-RECEIVING.rst;under its
ADMIN734_88/author-workspace/output:PUBLIC-EXPORT-VERIFIED04.json,verify-export-source04.log/.time,provenance-forward03.json,provenance-reverse03.json, correctedexport-case03/SOURCE-PACKET.jsonand originalruntime/mcp.{stdin,stdout,timing}.CSV reconciliation terminal0,0.05s,17100KiB RSS,Swaps0. Typed exact-owned
native close returned errors[] and acknowledged/process_exited/endpoint_terminated
alltrue for36406/create1791168723.62; independent PID/listener checks agree.
Original recorder terminal2 at23:25:20-04:00 retains rejected command history,
not shutdown failure: MCP4192764 independently absent and original scope
inactive/dead. Dewey owns the original7585 handoff/journal seal. All original
values/output evidence was retained before closure; no viewer was started.