Repository navigation
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Function references and registry entries state where a function comes from (distribution, version, imports; source hash for custom functions). Declared path kinds say what content a run reads. RunRecord builds, reads and rebuilds the record. PolyStore 0.6.3 (file_content) pinned. Not wired into the compiler, the workers or the metadata writer yet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The compiled bundle carries a RunDeclaration (complete pipeline document, saved global config document, resolved configuration with sources). Workers report the producer identity of each saved file. The parent hashes declared input files once before workers start and the metadata writer stores the record and the produced files in the dataset document's one transaction. Complete rendering (clean_mode=False) now writes the signature default of every function parameter a pattern leaves out, except those the platform supplies; range and module-exported callable defaults get formatters. The record stores the saved global config (global_config_code), not the effective config: rebuilding then resolves through the same hierarchy. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reads the record of what produced an output dataset, or of one output file, through RunRecord. Long texts are returned as hash and length unless asked for. The complete pipeline text writes every parameter; the authoring round-trip test states that for the full rendering. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cord's pipeline text is its own rendering Item 1. RunRecord, StepRecord, StepCalls, FunctionRecord, RecordedParameter, DeclaredFile, SoftwareEnvironment, ConfigValuesFromOwner, DatasetRunRecords replace the nested dicts. A parameter's source and a configuration value's source are registered class families stored as their registry key. The stored JSON is python_introspect.to_jsonable of the record and is read back with python_introspect.dataclass_from_mapping; the identity and file-content families are read as unions of dataclasses. Resolved configuration is grouped by the owner that supplied it, so each scope id is written once. The reader and the openhcs_read_run_record capability return these types; the capability returns what each step ran by default and the whole record on request. Item 2. clean_mode=False keeps its meaning (every stored field, no parameter the pipeline does not state). The record's copy is a separate rendering, PipelineDocumentCodec.render_with_every_parameter. The two tests that were weakened for the changed flag are restored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The function identity and the file content of a record are registered families
(defined_by: installed_module / source_text; content_kind: hashed / not_hashed)
and the record's fields are declared by the family, not by a union. The codec
writes {registry key: fields} and reads the class from the key; it no longer
picks a member by the fields present.
UiLiveMeasurementEntryState imports its preview type at run time: the codec
now reads declared field types when encoding.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…wo oversized classes are back to main's size - ProducedFiles.stated_by_step replaces OpenHCSMetadataTarget.saved_path_producers. - RunRecord.store_with_saved_files and RunRecordWriter (an AtomicMetadataWriter) store the record in its own locked update right after the dataset's metadata is published. OpenHCSMetadataTarget.finalize_completed_dataset and AtomicMetadataWriter.reconcile_completed_dataset are main's again. - The compiler builds the bundle of an empty compilation through one factory, CompiledExecutionBundle.with_nothing_to_run. - StepExecutionObservation.is_empty is derived from the dataclass fields. - The attribute that holds a custom source namespace is named in one place. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…line, what remains) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lands after DL1 and after four library releases.
python-introspect>=0.3.0,<0.4is pinned (0.3.0 is on PyPI: family members are stored under a tag). ObjectState, formbridge, pyqt-reactive and zmqruntime declarepython-introspect<0.3; each needs a release widening that bound before this branch can be installed.refactor/DL1-device-and-memory-typesmerges. Pins to set then:polystore>=0.6.4,<0.7,arraybridge>=0.5.0,<0.6, submoduleexternal/PolyStoreat the 0.6.4 tag (it points at5301d35of the Transport mode fix #41 branch until then;pyproject.tomlstill sayspolystore>=0.6.3,<0.7, which is wrong).State after the third hand-off: stored family members name their class (step A) and the record's owner stores it, with both oversized classes back to main's size and local guardrails at rc=0 (step B). Not started: the recorded-content defects, PolyStore 0.6.4, the dataset-root reader and the acceptance re-run. Exact list: "Third hand-off" in the receipt.
DRAFT: handed off mid-job. The receipt
docs/refactor/consolidation-20261009/DL3-output-provenance.mdhas the design, what is implemented, the acceptance results and the ordered list of what remains.What this does
Gap 6 of the Cellpose/N2V2 investigation (#1230): an output dataset now says what produced it.
RunDeclaration(complete pipeline text with every parameter written, the saved global config, and the resolved configuration with the scope and class ObjectState reported for each value). It travels on theCompiledExecutionBundle.RunRecord: per step the function identity (registry key, distribution and version, imports; source hash and text for a custom function), every parameter with its effective value and source (pipeline,function_default,run_time), assigned devices and memory types, and each declared file with the content hash the storage layer reports (PolyStorefile_content, 0.6.3: backends state a file's content hash and size, or why they cannot PolyStore#41).openhcs_metadata.jsonunderrun_records, in a locked update right after the dataset's metadata is published, withproduced_filesmapping each saved path to the producer identity a viewer item already carries.RunRecord.for_dataset,RunRecord.for_output_file,RunRecord.rebuild(); agent capabilityopenhcs_read_run_record(MCP tool andread-run-recordCLI command).State
tests/unit/test_run_record.py(17),tests/unit/test_run_record_capability.py(7),tests/integration/test_run_record_round_trip.py; round trip gives byte-identical TIFF and CSV outputs.guardrails_local.sh: rc=1 (ratchet rose: the record is handled as string-indexed dicts; code was added insidePipelineCompilerandOpenHCSMetadataTarget). Not yet fixed.smoke_local.sh, parity.DL3 · PR draft · guards 0/1 (guardrails failing) · not ready to merge
🤖 Generated with Claude Code